https://orcid.org/0000-0002-9695-5316
Figures
Abstract
Successful partnerships between people with different backgrounds and distinct beliefs are essential to solve 21st-century problems. We conducted 44 semi-structured interviews and 80 surveys with participants of academic-nonprofit partnerships. We used Plackett-Luce statistics and cultural consensus analysis to analyze respondent beliefs regarding partnership success. Respondents consistently placed a high value on a subset of ‘core’ reasons that academic-nonprofit partnerships succeed and exhibited higher-than-expected levels of agreement regarding the order of importance of these ‘core’ reasons. The approach we present here can be used before, during, and after partnerships to improve the design, implementation, and evaluation of team science.
Citation: Mathews MC, Drake AJ (2026) What makes partnerships work? Measuring agreement in participant beliefs regarding ‘core’ reasons for success. PLoS One 21(9): e0356371. https://doi.org/10.1371/journal.pone.0356371
Editor: AKM Alamgir, Access Alliance Multicultural Health and Community Services: Access Alliance, CANADA
Received: January 5, 2026; Accepted: August 3, 2026; Published: September 18, 2026
Copyright: © 2026 Mathews, Drake. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Deidentified survey data, R code utilized for analyses, and datasets produced in UCINet and an explanation of how they were created are available at: https://doi.org/10.6084/m9.figshare.33257574.
Funding: This research was supported in part by the Virginia G. Piper Charitable Trust. The conclusions, views, and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Virginia G. Piper Charitable Trust. This work was also funded by an Institute for Social Science Research (ASU) seed grant.
Competing interests: The authors have declared that no competing interests exist.
Introduction
As hubs of innovation and knowledge creation, academic institutions are increasingly expected to collaborate with partners in industry, government, nonprofit, and community sectors to solve complex problems. Millions of dollars and thousands of hours of research time have been invested in cross-sectoral and cross-disciplinary research programs focused on innovation and knowledge production [1,2]. These resources can be squandered if the partnerships upon which these efforts are built are poorly conceived, designed, and implemented.
A recurring finding in academic partnership research is that different work and organizational cultures can affect partnership outcomes [3–6]. For example, there is evidence that ‘ivory tower’ academics and ‘real world’ people in nonprofits, industry, schools, and government struggle to collaborate due to conflicting values, goals, and methods, which can “unnecessarily inhibit the forging of strong work relationships” [7]. Here, we measure the degree to which people participating in academic-nonprofit partnerships share beliefs regarding what causes these partnerships to succeed.
In this analysis, we address the following research questions:
- (i) What value or ‘worth’ do academic-nonprofit partnership participants place on the different steps, processes, and activities that lead to partnership success?
- (ii) Do the participants in academic-nonprofit partnership share beliefs about the order of importance of the steps, processes, and activities that lead to partnership success?
- (iii) Do subgroups of academic-nonprofit partnership participants hold distinct shared beliefs regarding the order of importance of the steps, processes, and activities that lead to partnership success?
Literature review
There is an extensive body of scholarly research regarding academic partnerships with communities, industry, nonprofits, and government organizations [8–10]. Scholars have produced numerous studies (and review articles) regarding how and why these partnerships succeed and fail [9,11,12]. Several studies focus on how organizational cultures can shape partnership outcomes [13–15]. Building on this work, we set out to measure the degree to which participants share beliefs regarding why academic-nonprofit partnerships succeed.
The importance of partnerships in 21st-century research.
Partnerships that bring together the specialized skills, knowledge, and resources of people from academia, industry, nonprofits, governments, and communities are essential to solving the complex ‘wicked’ problems of the 21st century [16,17]. These problems include, but are not limited to, climate change, biodiversity loss, homelessness, and income inequality.
Since the 1990s, academic institutions have faced increasing demands to go beyond traditional research and teaching to engage in ‘third mission’ activities that address complex societal problems [18,19]. ‘Third mission’ activities and research require non-academic knowledge and skills and depend upon partnerships that involve local communities, industry, nonprofits, and governments [20].
Convergence research is a 21st-century paradigm that involves partnerships between people and organizations with distinct backgrounds that span disciplinary boundaries [21]. Combining their unique skills and expertise enables convergence researchers to create innovative solutions for ‘wicked’ problems, such as climate change mitigation [22], community resilience [23], and hazards and disaster management [24], that require experts from different disciplinary areas.
Efforts to understand the dynamics of increasingly diverse research groups have coalesced into a research effort known as the Science of Team Science (SciTS). SciTS studies have shown that interdisciplinary teams are more likely to be highly productive and generate novel, high-impact research than lone scholars and teams with homogeneous backgrounds [25–27].
‘Third mission’, convergence, and the multi, inter, and transdisciplinary research studied by SciTS scholars depend on effective partnerships involving diverse actors with distinct backgrounds, motivations, and belief systems. The degree to which participants acknowledge, respect, and understand one another’s culture and belief systems can profoundly shape partnership outcomes.
Culture in partnerships.
When people share beliefs, values, and norms, they are said to have a common ‘culture’ [28]. Shared cultural models play a pivotal role in people’s understanding of the world and their behavior in it [29]. The role of culture in shaping partnership outcomes is a common thread across the different disciplines that study partnerships [15,30–32].
Although team diversity is considered essential to solving complex problems, it does not come without its own set of challenges. Experts with diverse backgrounds and expertise may have very different beliefs regarding how partnerships should work. Cultural differences can cause clashes regarding partnership goals, objectives, time schedules, activity calendars, and desired products and outcomes [33].
People involved in academic-nonprofit partnerships may come from different organizational cultures [34], with diverse workplace incentives and behavioral norms, speak different technical languages, and may even believe stereotypes about, or harbor resentment towards, one another [30,35]. Subcultures within groups may also have different belief systems and motivations. For example, students, staff, and professors belonging to the same academic culture may have distinct subgroup beliefs regarding partnerships [36].
It is often assumed that people understand partnerships in similar ways, such as opportunities to ‘work together’ [37]. However, people typically bring both their individual and organizational cultural models to partnerships [30]. These combinations of cultural models can lead to diverse beliefs regarding the importance of the different interactions, steps, and processes necessary for effective collaboration. If participants hold different beliefs regarding these processes, emphasizing some while deemphasizing others may lead to conflicts that negatively affect the partnership.
Distributed and team cognition in partnerships.
Cultural beliefs are just one type of shared cognition that can shape partnership outcomes. Team cognition theory argues that cognition occurs at the team level, where members collectively develop and share understandings, knowledge, and expertise, which they use to achieve collective goals and objectives [38,39]. In teams and partnerships, team cognition shapes members’ capacity to acquire, distribute, store, and retrieve critical knowledge [40]. Distributed cognition theory argues that cognitive processes, such as thinking, memory, and beliefs, are not restricted to individuals but are system-level processes shared across groups of people, environments, tools, and time [41].
Team and distributed cognition are manifested through shared mental models (SMMs) [42]. Team performance scholars use SMM theory to explain team dynamics and outcomes [43]. Gevers et al. [44] found that when team members shared beliefs about their collaborative tasks and timelines, it boosted their collective confidence and enhanced team performance. Measuring and understanding SMMs in teams and partnerships can help decipher how members share crucial information, where in the team unique knowledge resides, and how members act as a unit rather than as individuals [45].
Partial and overlapping shared mental models.
In some circumstances, people who do not share complete mental models may share beliefs about subcomponents of these models. For example, using cultural consensus and residual agreement analysis, Dengah [46] found that while people from two Pentecostal sects in Brazil largely agreed on some ‘core’ elements of what constituted a ‘complete life’, their beliefs diverged on the importance of others.
People engaging in partnerships may share subsets of beliefs that shape their behaviors in important ways, even when they do not share complete mental models. In our example here, ‘core beliefs’ refer to a subset of elements that a cross-section of people consistently deemed important for academic-nonprofit partnerships to succeed. Identifying these ‘core’ beliefs can serve as a starting point to define partnership goals and to design, plan, and implement the interactions, steps, and processes necessary for partnership success. Here we present an approach that can be used to identify shared beliefs and subsets of shared ‘core’ beliefs regarding elements that may be essential for partnership success.
Data collection
Our research effort included six phases of mixed methods data collection. This section explains each phase in detail, including our decision-making processes and how these steps and decisions shaped our final results.
Phase 1 – Literature review. Beginning in 2020, we conducted an extensive literature review of peer-reviewed articles, books, and grey literature, including how-to guides, focused on academic-nonprofit, academic-industry, academic-government, and academic-K12 partnerships. Our literature review indicated that although some partnership beliefs span the different types of academic partnerships, some beliefs may not. We focused our primary data collection on academic-nonprofit partnerships to avoid a potential mix of belief systems associated with distinct partnership types.
Phase 2 – Cultural domain analysis (Semi-structured interviews). In Phase 2, we conducted semi-structured interviews to conduct a cultural domain analysis of an emic/insider understanding of how academic-nonprofit partnerships work. Following Weller [47], we designed the cultural domain analysis to inform the development of our cultural consensus analysis survey in phase four. For example, we asked informants to freelist reasons for partnership success and then explain their importance for partnerships as suggested by Borgatti [48].
Following Handwerker’s [49] sampling guidelines for cultural domain analysis research, we used a purposive, snowball sampling approach to recruit people with academic-nonprofit partnership experience. We employed Maximum Variation Sampling [50] to capture a wide range of partnership beliefs. For example, we interviewed students, staff, and professors, as well as nonprofit CEOs, directors, and staff. Between August 2021 and June 2022, we conducted forty-four semi-structured interviews via Zoom, which we recorded and transcribed. Prior to being interviewed, all participants received the project consent form via email, which we reviewed again with the participants before each interview began. All participants provided verbal consent before recording and questions commenced. This consent process followed the protocol approved by the Office of Research Integrity and Assurance at Arizona State University. This study was determined to be exempt pursuant to Federal Regulations 45CFR46 [2].
Phase 3 – Qualitative coding and thematic analysis. We coded the semi-structured interview transcripts in MAXQDA. First, we consolidated the respondents’ freelists and answers for each question. Second, we grouped the similar list items and answers into themes. Finally, we created a comprehensive list of all the potential question responses from our semi-structured interviews and literature review. Our coding analysis enabled us to turn qualitative partnership expert’s cultural domain data into a list of reasons academic-nonprofit partnerships succeed. S1 Table A includes our final list of the reasons academic-nonprofit partnerships succeed and the number of semi-structured interview participants who mentioned each reason. This table also includes examples of publications from our literature review that contain mentions of the reasons for partnership success that people freelisted in the semi-structured interviews.
Phase 4 – Survey development and cognitive interviewing. In Phase 4, we developed an online survey instrument to measure shared beliefs regarding the themes and topics we identified in Phases 1–3. Our literature review and semi-structured interviews indicated people’s partnership beliefs might be heavily influenced by their organizational culture (academic vs nonprofit), their organizational status (student vs non-student), and their experience working on academic nonprofit partnerships. For this last trait, we asked three partnership experience questions, 1) the number of academic nonprofit partnerships in which respondents had participated, 2) the number of years they had worked on these partnerships, and 3) the types of organizations they worked in when engaged in these partnerships. During our semi-structured interviews, we discovered that people had often worked on partnerships in academic organizations and nonprofits at different stages in their careers. Therefore, we added a question asking if respondents had experience working on partnerships in academic organizations, nonprofits, or both. Table 1 describes the respondent attribute data from our survey.
Once the preliminary survey was completed, we conducted cognitive interviews via Zoom with a six-person panel over three sessions between January and April 2023. Following Willis [51], we created a protocol of cognitive interview questions to determine how the panelists understood each survey question and the descriptions of the rank-order items. Cognitive interviewees received IRB-approved consent form emails and provided verbal consent prior to the interviews. After the third round of cognitive interviews, we concluded that we had improved the survey to the best of our ability.
Phase 5 – Survey data collection. Our final survey contained seven rank-order questions regarding partnership beliefs and the aforementioned demographic and partnership experience questions. The latter were collected as potential explanatory variables that might be associated with shared beliefs regarding partnerships and are summarized in Table 1. We conducted our survey via the Qualtrics online survey platform. The list of rank order items was randomized in Qualtrics for each survey to avoid instances of false agreement. Respondents used the interactive features in Qualtrics to drag items into the order of their choosing.
Since cultural consensus analysis is used to measure cultural patterns of shared beliefs, respondents must have some experience with the cultural domain being modeled [47,52]. Therefore, scholars use non-random, purposive samples for CCA survey data collection [49,53]. Using a purposive sampling approach, we identified survey respondents who had participated in at least one academic-nonprofit partnership. We used Maximum Variation Sampling [50] to obtain a survey sample representative of the different subgroups that participate in academic-nonprofit partnerships, including students, staff, professors, and nonprofit CEOs. Our recruitment emails were approved by the Office of Research Integrity and Assurance at Arizona State University, as was the research explanation and consent statement at the beginning of the survey. Between May and October of 2023, we collected 80 online surveys via Qualtrics.
Phase 6 – Survey data analysis. The final phase included cleaning the completed survey data, creating a codebook, and conducting statistical analyses. All data in this study were deidentified to protect the participants’ anonymity. The methods section below explains our choice of statistical methods and how they work. In the results section, we present our research findings for the rank-order question in our survey focusing on the reasons partnerships succeed.
Methods
We used several statistical measures, including Plackett-Luce statistics, in conjunction with cultural consensus analysis (CCA) to analyze respondents’ answers to our ranking question about the causes of academic-nonprofit partnership success. Our goal with this multi-step analysis analytical triangulation was to determine if any patterns of partnership beliefs appeared across different statistical tests.
Plackett-luce statistics: Measuring preferences
The Plackett-Luce Model (PLM) combines the work of Luce [54] and Plackett [55]. Luce’s choice axiom states that the relative odds of choosing one item over another from a group of items are not affected by the presence or absence of the other group items [54]. Plackett [55] defined probability distributions for rank-order data, introduced parameters to characterize the probability distributions, and developed a parameter estimation method.
Based on Luce’s axiom, the rank ordering of the items is seen as a sequence of choices from the items remaining [56]. When an individual ranks a set of items from most to least important, they select the most important item from those that have yet to be ranked [57]. The Plackett-Luce statistic measures each item’s ‘worth’ parameter in a set of rank-order items based on how respondents rank them [58]. Items that respondents rank higher will also have a higher ‘worth’ value [59]. The Plackett-Luce model establishes a reference category to obtain the statistical significance of differences between item strength/worth [59]. This reference category can be one of the items or the mean item ‘worth’ [58].
Cultural consensus analysis (CCA): Measuring shared beliefs
While Plackett-Luce statistics are useful to identify item-by-item patterns in rank-order data, cultural consensus analysis (CCA) is used to determine if people share beliefs regarding answers to a set of questions [52,60]. Cultural consensus methods can be used to understand widely held cultural beliefs, such as what constitutes a ‘good life’ in Brazil [61], as well as shared beliefs among subgroups, such as how different Brazilian Pentecostal denominations understand what it means to be a ‘true believer’ [46]. Management and organization studies scholars have used CCA to measure shared knowledge and cultures in organizations [62], to identify ‘operational subcultures’ [63], and to compare international beliefs regarding cooperation [64].
Cultural consensus analysis measures shared beliefs by first creating an agreement matrix of the correlations of responses to survey questions [52]. Each matrix cell captures the correlation between two respondents’ answers. Next, eigenfactor decomposition (similar to factor analysis) is run on the agreement matrix to determine if there is a strong agreement in the form of a single-factor solution [52]. Next, a competence score is calculated for each respondent by comparing their answers with those of the group [47]. This score, typically between 0 and 1, captures the percentage of each respondent’s answers that agreed with the group’s ‘culturally correct’ answers [65]. The model also produces a 2nd factor loading score for each respondent, capturing variation beyond the first factor [47]. These scores can be used to understand residual agreement, which may exist between members of respondent subgroups [66]. Finally, the model produces an answer key by weighting each respondent’s answers by their competency score [52]. The answer key is deemed to represent the most agreed-upon answers to the CCA questions [67]. By convention, three diagnostics indicate if shared beliefs exist among respondents: 1) the eigenvalue ratio between the first and second eigenvalues is greater than 3:1, 2) there are few, if any, negative competence scores, and 3) there is a mean competence score of 0.5 or greater [47].
Cultural consensus models are calculated differently depending on the type of questions used to collect data. The informal model accepts ranked, interval, or ratio-scaled response data [47]. In this model, an agreement matrix is factored, and the competence scores are used to weight respondent’s responses (by multiplication), which are then summed together [47].
We decided that multiple-choice, Yes/No, or True/False questions might not be very illuminating. For example, measuring whether people agree that effective communication is important for partnership success might not advance our understanding of partnerships. Instead, we used a rank-order question format to determine whether respondents shared beliefs regarding the importance of reasons, like effective communication, compared to other reasons for success. Fig 1 shows our research workflow, which includes data analysis methods that when used together, make it possible to identify shared beliefs and subsets of ‘core’ beliefs. We have not seen this set of statistical methods used together to determine the ‘worth’ people place on reasons for partnership success and the degree to which groups share these beliefs.
This figure provides a visual of project workflow.
Results
In this section, we present the results from our analysis of question 10 in our survey, which asked respondents to rank order the reasons academic-nonprofit partnerships succeed. In this question, respondents were asked to rank order the 18 items in S1 Table A from most to least important as reasons for academic-nonprofit success. The reasons are also listed in Table 2.
- a. Statistical analyses of ranking hierarchies
We used Kendall’s coefficient of concordance to measure the consistency of rankings across the 80 survey respondents, Kendall’s W(80, N = 18) = .254, χ2 (17) = 345.7, p < 2.2e-16. The results indicated a level of ‘fair agreement’ according to the interpretation levels suggested by Landis and Koch [68]. However, it is important to note that Kendall’s W tests absolute, group-wide concordance in exact ranking and may not capture agreement regarding subgroups of ranking items.
We conducted a Friedman rank sum test to determine if respondents showed a significant preference among the 18 items. The results indicated a statistically significant difference in the rankings across the items, χ2 (17) = 345.7, p < 2.2e-16. Since the Friedman test is global, we also ran a post hoc pairwise comparisons Conover test with a Bonferroni correction. The p-value table from this test (S1 Table B) shows which ranking pairs had means that were statistically different (<0.05) and which did not (>0.05).
Pairing the pairwise comparison p-value table (S1 Table B) with the mean rankings (Fig 2), revealed a definitive hierarchy in the importance respondents placed on different reasons for academic-nonprofit partnership success. Q10b, Q10k, and Q10l were highly unique; they had extremely small p-values <0.05 (S1 Table B), compared to the other reasons for success, indicating that respondents ranked them differently. These reasons also had the lowest mean rankings (Top Tier in Fig 2), indicating they were consistently ranked as more important for partnership success. A marginal ranking table (S1 Table C) corroborates this result, Q10b was ranked in the top three by 46 out of the 80 respondents (58%) and Q10l was ranked number one by 29 out of the 80 respondents (36%).
This figure highlights mean comparisons for reasons for partnership success based on survey results.
Together, the post hoc p-values and mean ranks suggest Q10o and Q10q sit in a Mid-high Tier (Fig 2) of reasons that were considered less important than the top three, but more important than the rest of the reasons. The Middle Cluster (Fig 2) includes reasons that were largely statistical ties when compared to one another in the pairwise analysis, indicating a lack of agreement regarding which were more important. The marginal ranking table (S1 Table C) corroborates this finding, Q10h for example, received similar votes for all the ranking choices. Finally, there was a Bottom Tier (Fig 2) of reasons that were statistically indistinguishable from each other in the pairwise comparisons and had the highest mean ranks, meaning they were consistently ranked lower than the other reasons (S1 Table B).
The statistical tests in this section indicate that respondents shared mental models regarding the ‘core’ importance of communication, vision, and goals reasons for success (Q10b, Q10l, and Q10k) while agreeing slightly less regarding the importance of establishing clear roles and obtaining sufficient funding (Q10o and Q10q).
- b. Plackett-Luce item ‘worth’
The summary in Table 2 includes the ‘worth’ estimates, standard errors, and associated hypothesis test statistics and p-values for each item when the mean ‘worth’ is used as the reference. Here, a higher item ‘worth’ indicates that a reason was more likely to be ranked higher as something that leads to partnership success. The top five reasons (Q10b – Q10q) had statistically significant positive estimates that indicate that they were all ranked higher than the mean ranking (9.5). The following five reasons (Q10e – Q10c) in Table 2 had statistically non-significant positive and negative estimates that were neither above nor below the mean ranking. The bottom eight reasons (Q10p – Q10r) had statistically significant negative estimates, indicating they were consistently ranked lower than the mean ranking. Lower ranked items should not be interpreted as unimportant for partnership success. Our rank order question required respondents to prioritize reasons for success, revealing patterns in which items were generally given a higher or lower priority as reasons for success rather than distinguishing between important and unimportant factors.
These Plackett-Luce estimates and Z-values suggest that respondents believed that the top four items in Table 2 (Q10b, Q10k, and Q10l) were the most important reasons for partnership success. These reasons centered on clear goals and objectives, effective communication, and shared vision/purpose. Rounding out the top five were clear roles/responsibilities (Q10o) and sufficient funding (Q10q). This Plackett-Luce item ‘worth’ analysis illuminates respondents’ value systems regarding the individual reasons for partnership success. While the differences between the top five reasons were modest, the differences between the top and bottom reasons in Table 2 were significant. The Plackett-Luce estimates, z-values, and probability of ranking first measures (Table 2) corroborate the findings of the Friedman rank test, marginal ranking table, and post hoc pairwise comparisons, revealing a clear preference for the Top Tier and Mid-High Tier reasons for success in Fig 2. They refine the analysis of the Middle Cluster by comparing the ranked items to the mean center (9.5), which separates items in the neutral center from those trending downwards. This explains the difference between the Middle Cluster in Fig 2 and Table 2. The Plackett-Luce results also reiterate the levels of agreement regarding reasons that were consistently ranked less important for partnership success in the other tests (Bottom Tier Fig 2).
We can evaluate the performance of the Plackett-Luce model (PLM) using the residual deviance statistic [58], which for this model was 5485.2 with 12,223 degrees of freedom. The deviance-to-df ratio was 0.45. A deviance-to-df ratio well below 1.0 implies underdispersion in the model, meaning that there are fewer differences between the observed data and fitted probabilities than expected [69]. This may be due to the strong agreement across the sample of the importance of the Top Tier reasons for partnership success that were especially evident in the marginal ranking table, mean ranks, and post hoc pairwise comparisons. However, we also recognize that our sample size and sampling approach may have implications for our statistical results (see limitations section below). Nevertheless, given that the PLM estimates largely agree with the mean rankings and pairwise comparisons, the model appears to capture the basic hierarchical trends in the respondents’ partnership success beliefs.
Cultural consensus analysis (CCA) and shared partnership beliefs
- a. Shared partnership beliefs and residual agreement among subgroups
We used cultural consensus analysis to test whether the results of respondents’ rankings exhibited patterns of shared beliefs regarding the order of importance of reasons academic-nonprofit partnerships succeed. First, it is important to note the likelihood of respondents ranking the reasons for success in the same order. The possible rankings for these items are 18 factorial (18!), which equals a staggering 6.402374e + 15 or 6.4 quintillion possible ranking combinations. With 80 respondents and such a large number of possible ranking orders (1/18!)^80, the likelihood of 80 respondents ranking all 18 items the same by chance is astronomically small. With so many possible ranking options and such low probabilities of ranking items in the same order by chance, we expected little if any agreement in the rankings.
The ratio between the first and second eigenvalue for the cultural consensus model with all 80 respondents was 2.96, with a mean competence score of .49, and zero negative competencies. We used UCINet 6 Version 6.801 to run the consensus models [70]. Although the model meets the suggested minimum threshold of 3:1 for eigenvalue ratios and >.50 for competence scores [47,71], it is important to note that this model hovers right at the minimum threshold. Eigenvalue ratios below 3:1 may indicate that there is no single, shared cultural model among the respondents, or that subgroups within the sample hold distinct or conflicting beliefs [72]. Given this possibility, we tested whether subgroups in the sample exhibited higher agreement levels.
T-tests with the binary demographic variables from Table 1 and the respondents’ competence scores indicated that some subgroups had different levels of agreement with the overall sample of respondents (see Table 3). The t-test for gender did not reveal a significant difference in the mean competence scores for men and women. We removed the single non-binary respondent from this analysis. The ANOVAs for Organization Type and Partnership Experience Type were not statistically significant, indicating that the levels of agreement of respondents in these subgroups regarding reasons for partnership success did not vary widely from the overall sample.
However, the t-tests for the student status and partnership experience variables identified statistically significant differences in mean competence scores for these groups (Table 3). Non-students had higher mean competence scores, indicating that their rankings were more strongly aligned with the shared beliefs of the entire respondent group. The Cohen’s d effect for this test was large, suggesting that students and non-students understood the reasons partnerships succeed differently. This result may have several causes and implications. First, the nature of our sampling approach (purposive snowball), led to a sample of students of different age groups, partnership experience, and disciplinary focus areas. This diversity may have led to the lack of agreement in the student subgroup. Answering questions regarding differences in beliefs between students and non-students requires replicating this research in different settings (see next steps below).
T-tests for the number and years of partnership experience (Table 3) indicated that respondents with more partnership experience had higher average competence scores, which suggests that their rankings were more strongly aligned with those of the entire respondent group. However, the Cohen’s d effects for these t-tests were medium, the p-values were not strongly significant, and the difference in the means was not large.
- b. Residual agreement results
Since the competence scores indicated the possibility of residual agreement beyond the first factor, we ran several residual agreement tests. First, following Boster [73], we calculated a predicted agreement matrix for the entire sample and subtracted it from the observed agreement matrix to generate a residual agreement matrix. We then ran a QAP correlation of the observed and residual matrices. The correlation was .752 with a p-value of 0.000. When the observed and residual agreement matrices are correlated, the overall group consensus does not exhaust the residual agreement, and subgroup belief variations may be present [66].
Next, following Boster and Johnson [74], we plotted the competence scores and the 2nd factor loading scores (S1 Fig–S6 Fig). We generated plots for each respondent attribute collected in the survey that may explain subgroup variation. To interpret the plots, we turn to the residual agreement framework created by Henderson et al. [72]. Given that the respondent competence scores (S1 Fig–S6 Fig) cluster along the 1st factor (x-axis) and are dispersed along the 2nd factor (y-axis), we appear to have a multicentric cultural domain with overall consensus [72]. This means that while there is overall group consensus (as indicated by the 2.96 eigenvalue for the overall data), the agreement pattern tends to be stronger within residual agreement groups [72].
To better understand the subgroup consensus, we first ran cultural consensus models for each of the subgroups. Table 4 contains the cultural consensus results for subgroups in our data that exhibited some shared beliefs. First, we can see that the eigenvalue ratios all crossed the 3:1 threshold, with mean competence scores > .50 threshold. The competence score standard deviations were not small, indicating that agreement was unevenly distributed, even within the context of overall agreement. The range of the competence scores was wide, indicating variation between the beliefs of some respondents and the overall group. Finally, there was only one negative competence score, indicating that only one respondent disagreed with one model.
To better measure agreement across subgroups, we compared the answer keys from all the subgroup CCA models to determine if (and how) they were similar. First, we created a heat map of the subgroup answer keys for side-by-side comparison (Table 5), which reveals several consistent patterns. First, the heat map color intensity shows similar high rankings across answer keys for the ‘core’ reasons from the Top Tier and Mid-High Tier reasons in (Fig 2. And Table 3), which were consistently ranked the most important. Second, we also see similar rankings across answer keys for the lowest-ranked reasons. Third, the heat map shows that the most divergent answer key belonged to the students, who disagreed with the other subgroups regarding the importance of administrative buy-in (Q10d) and partnership champions (Q10p). Although the answer keys varied across some rankings, which explains the different competence scores among subgroups, the ‘core’ reasons for success remained remarkably consistent across subgroup answer keys.
Notably, the answer key analysis results mirrored the semi-structured interview data; the top three reasons for success were the most frequently free-listed (S1 Table A) and consistently ranked the highest. We also ran a Kendall’s Tau correlation of the answer keys, which revealed a great deal of agreement across the answer keys, except for the student answer key (S1 Table D).
Finally, following Dengah and Dressler [65,66], we created divergence plots to determine the degree to which the rankings of the reasons for success varied across subgroups (S7 Fig–S14 Fig). In this approach, positive subgroup rankings have higher deviation scores (e.g., 1,2,3) in the plot, and lower-ranked items have lower deviation scores (e.g., −1, −2, −3) [46]. Items with no deviation between the subgroup and overall respondent sample rankings lie at the plot origin (0,0) [46]. We created plots for each of the different subgroups in Table 4. In these plots, we colored the Top Tier ‘core’ reasons for success in red and the Mid-High Tier in pink. Although there is some variation across subgroups (particularly the student vs non-student plot), the Top Tier ‘core’ reasons hovered around the origin in each plot, which reconfirms the finding that the Top Tier (communication, vision, and goals) reasons were consistently ranked highly as important for partnership success. The Mid-High Tier reasons fluctuated much more, indicating less consistency in respondent’s shared mental models regarding clear roles (Q10o) and sufficient funding (Q10q) than those regarding the Top Tier reasons for partnership success.
Research limitations
A mix of statistical analyses are useful to verify the existence of patterns, such as the consistent importance of ‘core’ beliefs, in data. However, challenges can arise when statistical approaches have different assumptions regarding sampling and other data characteristics. Given that our cultural consensus survey questions required a non-random, purposive sample, we do not make any claims regarding the inferential power of our statistical measures to explain wider beliefs regarding partnerships beyond our sample. This will require replication, which we address in the discussion section. In addition, while 80 respondents may be more than adequate for many cultural consensus analysis studies, a larger sample may be necessary for a rank order question with 18 items to produce robust statistical findings for the other analyses used here.
We conducted this research while working at a university which emphasizes partnerships with non-academic organizations, prizes community engagement, and has offices that help connect academics with community partners. Since these conditions may have shaped our research outcomes, we believe it is necessary to replicate this research in other settings to determine if our findings are more widely generalizable (see discussion).
Results summary
In this study, we used semi-structured interviews with academic-nonprofit partnership participants to generate a list of reasons partnerships succeed from domain experts. A combination of analyses including Friedman’s test, mean ranks, post hoc pairwise comparisons, and a marginal ranking table identified a set of ‘core’ reasons partnerships succeed. Plackett-Luce statistics captured the ‘worth’ that participants assigned to each reason for partnership success. Finally, we used cultural consensus analysis to measure partnership participants’ shared beliefs regarding the order of importance of the reasons academic-nonprofit partnerships succeed.
The statistical tests and CCA analysis answer keys all indicated that respondents placed a high value on communication, vision, goals, roles, and funding as reasons academic-nonprofit partnerships succeed. The residual agreement divergence plots indicate that these ‘core’ reasons for success were less likely to diverge in subgroup beliefs (S7 Fig–S14 Fig). These ‘core’ reasons for success appear to be foundational processes upon which partnerships can be built. For instance, once everyone agrees on what a partnership is about (Q10l), what everyone will do (Q10o), and when they will do it (Q10b), other steps can be taken. Clear communication (Q10k) is essential to organize the ‘what,’ ‘who,’ and ‘when,’ and to make adjustments as the partnership unfolds. Sufficient funding (Q10q) is also an important ingredient for success, although respondents’ beliefs regarding funding were not as consistently shared as their beliefs about the other ‘core’ items.
In response to the research questions at the beginning of this article, we can say the following. First, respondents in our sample consistently placed a high value or ‘worth’ on a ‘core’ set of reasons that academic-nonprofit partnerships succeed (Fig 2, Table 2). Second, respondent rankings exhibited higher-than-expected amounts of shared belief regarding the order of importance of a subset of ‘core’ reasons partnerships succeed. Third, although academic and nonprofit respondent subgroups from distinct organizational cultures ranked some reasons differently, they agreed with the overall respondent group regarding the importance of the ‘core’ reasons for academic-nonprofit partnership success.
Discussion
The innovation necessary to solve complex 21st-century challenges requires partnerships that bring together people with diverse skills, knowledge, and expertise. These people often come from distinct organizational cultures and may understand partnerships differently. The degree to which participants share beliefs and mental models regarding which steps, processes, and activities lead to success may shape partnership outcomes.
Our exploratory research indicates that people may share a subset of ‘core’ beliefs regarding how partnerships work, even when they do not share complete mental models. If this finding is repeated with additional research, it raises new questions that merit further exploration. For example, do beliefs regarding similar ‘core’ reasons for success exist across different partnership types? If so, is it enough for people to share ‘core’ beliefs in order for partnerships to succeed? How do we leverage this knowledge to design better partnerships?
Our research shows how mixed data analysis methods can be combined to measure shared mental models in partnerships. However, there are numerous possible next steps. This research could also be replicated with different types of academic partnerships (industry, K12, community, government) to determine if similar belief patterns emerge. In fact, this approach could be replicated across different types of partnerships (e.g., business, diplomatic, family, etc.) to determine if people share beliefs regarding a set of ‘core’ reasons that all partnerships succeed.
Upon reflection, an additional step of reviewing the statistical results with research participants could enhance this research. In this follow-up, researchers could ask participants why they ranked certain reasons for partnership success higher (or lower), whether they agree (or not) that certain steps are more important for partnership success, and why this is the case. These follow-up interviews can serve as an opportunity to return research results to participants and as a way to gain information that can help interpret the statistical results. For example, follow-up questions could be used to explore why respondents shared such strong beliefs regarding the importance of the Top Tier ‘core’ reasons for partnership success.
Our approach can be used as an evaluation technique before, during, and after partnerships to measure participants’ belief systems, set partnership goals, establish benchmarks for success, and plan partnership activities. For example, participants can use our approach to measure their shared beliefs at the outset of a partnership and then periodically throughout the partnership to determine if participants’ beliefs change. Participants can use these results to adjust partnership steps and activities as needed. This effort can then be replicated as part of a final partnership evaluation to determine how beliefs and mental models compare across the partnership’s life cycle. This approach is not limited to questions about reasons for success, it can be used to measure shared beliefs regarding other partnership design elements.
By sampling people with academic-nonprofit experience, we measured beliefs related to a larger distributed cognition regarding these partnerships. However, this approach did not capture the connection between shared beliefs regarding partnership success and successful partnership outcomes. In order to measure team cognition and determine if there is a linkage between shared beliefs and partnership success, this research would need to be replicated with individual partnerships over time. With longitudinal data across multiple partnerships, scholars could measure the association between shared partnership beliefs and the likelihood of partnership success. By understanding people’s shared beliefs regarding how partnerships work, we may be able to better use the scarce resources that make research possible and harness the forces that drive innovation and knowledge creation.
Supporting information
S1 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Gender).
https://doi.org/10.1371/journal.pone.0356371.s001
(TIFF)
S2 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Student Status).
https://doi.org/10.1371/journal.pone.0356371.s002
(TIFF)
S3 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Organization Type).
https://doi.org/10.1371/journal.pone.0356371.s003
(TIFF)
S4 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Experience Type).
https://doi.org/10.1371/journal.pone.0356371.s004
(TIFF)
S5 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Above Median Number of Partnerships).
https://doi.org/10.1371/journal.pone.0356371.s005
(TIFF)
S6 Fig. Q10 Respondent Cultural Competence and 2nd Factor Loading Scores (Above Median Number of Years Partnerships).
https://doi.org/10.1371/journal.pone.0356371.s006
(TIFF)
S7 Fig. Q10 Answer Key Divergence (Student vs Non-student).
https://doi.org/10.1371/journal.pone.0356371.s007
(TIFF)
S8 Fig. Q10 Answer Key Divergence (Academic vs Nonprofit).
https://doi.org/10.1371/journal.pone.0356371.s008
(TIFF)
S9 Fig. Q10 Answer Key Divergence (Academic Non-student vs Nonprofit Non-student).
https://doi.org/10.1371/journal.pone.0356371.s009
(TIFF)
S10 Fig. Q10 Answer Key Divergence (Academic vs Nonprofit Experience).
https://doi.org/10.1371/journal.pone.0356371.s010
(TIFF)
S11 Fig. Q10 Answer Key Divergence (Academic vs Both Experience).
https://doi.org/10.1371/journal.pone.0356371.s011
(TIFF)
S12 Fig. Q10 Answer Key Divergence (Nonprofit vs Both Experience).
https://doi.org/10.1371/journal.pone.0356371.s012
(TIFF)
S13 Fig. Q10 Answer Key Divergence (Above vs Below Median Number of Partnerships).
https://doi.org/10.1371/journal.pone.0356371.s013
(TIFF)
S14 Fig. Q10 Answer Key Divergence (Above vs Below Median Number of Partnership Years).
https://doi.org/10.1371/journal.pone.0356371.s014
(TIFF)
S1 Table A. Reasons for academic-nonprofit partnership success from literature review and semi-structured interviews.
This table contains our list of reasons for partnership success based on a literature review and semi-structured interviews.
https://doi.org/10.1371/journal.pone.0356371.s015
(DOCX)
S1 Table B. Post hoc pairwise comparisons Conover test with a Bonferroni correction.
Top-Tier and Mid-High Tier reasons in bold. This table provides the results of Conover test and the resulting p-values.
https://doi.org/10.1371/journal.pone.0356371.s016
(DOCX)
S1 Table C. Marginal ranking table of reasons for academic-nonprofit partnership success.
This table shows how often each reason for partnership success received each ranking.
https://doi.org/10.1371/journal.pone.0356371.s017
(DOCX)
S1 Table D. Kendall’s Tau correlation table of cultural consensus subgroup answer keys.
This table shows the intensity of the correlations of rankings across subgroup cultural consensus answer keys.
https://doi.org/10.1371/journal.pone.0356371.s018
(DOCX)
Acknowledgments
We would like to thank the interview participants and survey respondents whose insights and information made this research possible. Christina Ngo and the Office of Community Embeddedness at Arizona State University were incredibly helpful during our respondent recruitment. Daniel Hruschka and Susan C. Weller provided guidance during different phases of the research. François Dengah, H. Russell Bernard, and Kevin C. Lee kindly reviewed several drafts of this article. We also thank the anonymous reviewers for their excellent suggestions during the peer-review process.
References
- 1. Baru C, Campbell L, Dade A, Fulay P, Loewi A, Maughan D, et al. The NSF Convergence Accelerator program. AI Magazine. 2022;43(1):6–16.
- 2.
Cooke NJ, Hilton ML. Enhancing the Effectiveness of Team Science. Washington, D.C.: The National Academies Press. 2015.
- 3. Ehrismann D, Patel D. University – industry collaborations: models, drivers and cultures. Swiss Medical Weekly. 2015;145(0506).
- 4. Erosa VE. Dealing with cultural issues in the triple helix model implementation: a comparison among government, university and business culture. Procedia - Social and Behavioral Sciences. 2012;52:25–34.
- 5. Lacy WB, Glenna LL, Biscotti D, Welsh R, Clancy K. The two cultures of science: implications for university-industry relationships in the U.S. agriculture biotechnology. Journal of Integrative Agriculture. 2014;13(2):455–66.
- 6. Wasonga CO. Re-thinking school-university collaboration: agenda for the 21st century. Educ Res Rev. 2012;7(11).
- 7. Hill LW. Political and academic linkages in public sector policymaking. Journal of Higher Education Outreach and Engagement. 2006;11(1):221.
- 8. Ankrah S, AL-Tabbaa O. Universities–industry collaboration: a systematic review. Scandinavian Journal of Management. 2015;31(3):387–408.
- 9. Gazley B, Guo C. What do we know about nonprofit collaboration? a systematic review of the literature. Nonprofit Mgmnt & Ldrshp. 2020;31(2):211–32.
- 10. Gutierrez AS, Fox J, Clifton J. The significance of partnerships to future university missions: a systematic literature review. Innovations in Education and Teaching International. 2024;62(3):1101–15.
- 11. Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, et al. Community-academic partnerships: a systematic review of the state of the literature and recommendations for future research. Milbank Q. 2016;94(1):163–214. pmid:26994713
- 12. Rybnicek R, Königsgruber R. What makes industry–university collaboration succeed? a systematic review of the literature. J Bus Econ. 2018;89(2):221–50.
- 13. Eckel PD, Hartley M. Developing academic strategic alliances: reconciling multiple institutional cultures, policies, and practices. The Journal of Higher Education. 2008;79(6):613–37.
- 14. Kienast S-R. How do universities’ organizational characteristics, management strategies, and culture influence academic research collaboration? a literature review and research agenda. Tert Educ Manag. 2023;29(2):139–60.
- 15. Meirovich G. Cultural similarities and differences: impacts on performance in strategic partnerships. Advances in Mergers and Acquisitions. Emerald Group Publishing Limited. 2012. 55–74.
- 16. Gilbertson M, Craft M, Potter T. Planetary grand challenges: a call for interdisciplinary partnerships. Interdiscip J Partnersh Stud. 2019;6(1):1–17. pmid:33912682
- 17. Pohl C, Truffer B, Hirsch-Hadorn G. Addressing wicked problems through transdisciplinary research. The Oxford Handbook of Interdisciplinarity. Oxford University Press. 2017. 319–31.
- 18. Compagnucci L, Spigarelli F. The Third Mission of the university: a systematic literature review on potentials and constraints. Technological Forecasting and Social Change. 2020;161:120284.
- 19. Laredo P. Revisiting the third mission of universities: Toward a renewed categorization of university activities?. Higher Education Policy. 2007;20:441–56.
- 20. Rinaldi C, Cavicchi A, Spigarelli F, Lacchè L, Rubens A. Universities and smart specialisation strategy: From third mission to sustainable development co-creation. International Journal of Sustainability in Higher Education. 2018;19(1):67–84.
- 21.
National Research Council. Convergence: Facilitating Transdisciplinary Integration of Life Sciences, Physical Sciences, Engineering, and Beyond. Washington, DC: The National Academies Press; 2014.
- 22. Birthisel SK, Eastman BA, Soucy AR, Paul M, Clements RS, White A, et al. Convergence, continuity, and community: a framework for enabling emerging leaders to build climate solutions in agriculture, forestry, and aquaculture. Climatic Change. 2020;162(4):2181–95.
- 23. Sundstrom SM, Angeler DG, Ernakovich JG, García JH, Hamm JA, Huntington O. The emergence of convergence. Elementa: Science of the Anthropocene. 2023;11(1).
- 24. Peek L, Tobin J, Adams RM, Wu H, Mathews MC. A framework for convergence research in the hazards and disaster field: the natural hazards engineering research infrastructure CONVERGE facility. Front Built Environ. 2020;6.
- 25. Cotropia CA, Petherbridge L. The dominance of teams in the production of legal knowledge. The Yale Law Journal Forum. 2014;:18–28.
- 26. Uzzi B, Mukherjee S, Stringer M, Jones B. Atypical combinations and scientific impact. Science. 2013;342:468–72.
- 27. Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316(5827):1036–9. pmid:17431139
- 28.
D’Andrade RG. Cultural meaning systems. Shweder R, LeVine RA. Culture Theory: Essays on mind, self, and emotion. Cambridge, UK: Cambridge University Press. 1984.88–119.
- 29. Quinn N, Holland D. Culture and cognition. Cultural Models in Language and Thought. Cambridge University Press. 1987. 3–40.
- 30.
Hora MT, Millar SB, Ramaley J. A Guide to Building Education Partnerships: Navigating Diverse Cultural Contexts to Turn Challenge into Promise. New York: Routledge. 2011.
- 31. Jones G, Chirino Chace B, Wright J. Cultural diversity drives innovation: empowering teams for success. IJIS. 2020;12(3):323–43.
- 32. Stahl GK, Maznevski ML. Unraveling the effects of cultural diversity in teams: a retrospective of research on multicultural work groups and an agenda for future research. J Int Bus Stud. 2021;52(1):4–22. pmid:33487775
- 33. Hemmert M, Bstieler L, Okamuro H. Bridging the cultural divide: trust formation in university–industry research collaborations in the US, Japan, and South Korea. Technovation. 2014;34(10):605–16.
- 34. Hofstede G, Neuijen B, Ohayv DD, Sanders G. Measuring organizational cultures: a qualitative and quantitative study across twenty cases. Administrative Science Quarterly. 1990;35(2):286.
- 35. John MB. Norms and the Work of Colleges and Universities: Introduction to the Special Issue—Norms in Academia. The Journal of Higher Education. 2010;81(3):243–50.
- 36. Francis JE. The functions and norms that drive university student volunteering. J of Philanthropy and Mktg. 2011;16(1):1–12.
- 37. Montuori A, Conti I. The meaning of partnership. Vision/Action. 1995;14(4):7–10.
- 38. Grand JA, Braun MT, Kuljanin G, Kozlowski SWJ, Chao GT. The dynamics of team cognition: a process-oriented theory of knowledge emergence in teams. J Appl Psychol. 2016;101(10):1353–85. pmid:27504660
- 39.
Salas E, Fiore SM, Letsky MP. Theories of Team Cognition: Cross-Disciplinary Perspectives. New York: Routledge. 2012.
- 40. Fernandez R, Shah S, Rosenman ED, Kozlowski SWJ, Parker SH, Grand JA. Developing team cognition: a role for simulation. Simul Healthc. 2017;12(2):96–103. pmid:28704287
- 41.
Hutchins E. Distributed cognition. Smelser NJ, Baltes PB. International Encyclopedia of the Social & Behavioral Sciences. Amsterdam, Netherlands: Elsevier Ltd. 2001. 2068–72.
- 42. Levine JM. Socially-shared cognition and consensus in small groups. Curr Opin Psychol. 2018;23:52–6. pmid:29306838
- 43. Mohammed S, Ferzandi L, Hamilton K. Metaphor no more: a 15-year review of the team mental model construct. Journal of Management. 2010;36(4):876–910.
- 44. Gevers JMP, Li J, Rutte CG, van Eerde W. How dynamics in perceptual shared cognition and team potency predict team performance. J Occupat & Organ Psyc. 2019;93(1):134–57.
- 45. Wildman JL, Salas E, Scott CPR. Measuring cognition in teams: a cross-domain review. Hum Factors. 2014;56(5):911–41. pmid:25141596
- 46. Dengah HJF. The contract with god: patterns of cultural consensus across two Brazilian religious communities. J Anthropol Res. 2013;69(3):347–72.
- 47. Weller SC. Cultural consensus theory: applications and frequently asked questions. Field Methods. 2007;19(4):339–68.
- 48.
Borgatti SP. Elicitation techniques for cultural domain analysis. Schensul JJ, LeCompte MD, Nastasi BK, Borgatti SP. Enhanced Ethnographic Methods: Audiovisual Techniques, Focused Group Interviews, and Elicitation Techniques. Lanham, MD: Altamira Press. 1999. 115–50.
- 49. Handwerker WP. The construct validity of cultures: cultural diversity, culture theory, and a method for ethnography. Am Anthropol. 2002;104(1):106–22.
- 50. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2015;42(5):533–44. pmid:24193818
- 51.
Willis GB. Cognitive interviewing: a “how to“ guide. Research Triangle Park, NC: Research Triangle Institute. 1999.
- 52. Romney AK, Weller SC, Batchelder WH. Culture as consensus: a theory of culture and informant accuracy. Am Anthropol. 1986;88(2):313–38.
- 53.
Handwerker WP. Sample Design. Kempf-Leonard K Encyclopedia of Social Measurement. Amsterdam, Netherlands: Elsevier Science. 2005. 429–36.
- 54.
Luce RD. Individual choice behavior: A theoretical analysis. New York: Wiley. 1959.
- 55. Plackett RL. The analysis of permutations. Journal of the Royal Statistical Society Series C (Applied Statistics). 1975;24(2):193–202.
- 56.
Alvo M, Yu PLH. Statistical Methods for Ranking Data. New York: Springer. 2014.
- 57. Yu PLH, Gu J, Xu H. Analysis of ranking data. WIREs Computational Stats. 2019;11(6).
- 58. Turner HL, van Etten J, Firth D, Kosmidis I. Modelling rankings in R: the PlackettLuce package. Comput Stat. 2020;35(3):1027–57.
- 59. Finch H. An introduction to the analysis of ranked response data. Practical Assessment, Research, and Evaluation. 2022;27.
- 60. Batchelder WH, Anders R, Oravecz Z. Cultural Consensus Theory. Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience. Wiley. 2018. 1–64.
- 61. Dressler WW, Borges CD, Balieiro MC, dos Santos JE. Measuring cultural consonance: examples with special reference to measurement theory in anthropology. Field Methods. 2005;17(4):331–55.
- 62. Borgatti SP, Carboni I. On measuring individual knowledge in organizations. Organizational Research Methods. 2007;10(3):449–62.
- 63. Smith CS, Morris M, Hill W, Francovich C, Christiano J. Developing and validating a conceptual model of recurring problems in teaching clinic. Adv Health Sci Educ Theory Pract. 2006;11(3):279–88. pmid:16832710
- 64. Keller J, Loewenstein J. The cultural category of cooperation: a cultural consensus model analysis for China and the United States. Organ Sci. 2011;22(2):299–319.
- 65.
Dengah HJF, Snodgrass JG, Polzer ER, Nixon WC. Systematic Methods for Analyzing Culture: A Practical Guide. New York: Routledge. 2021.
- 66. Dressler WW, Balieiro MC, dos Santos JE. Finding culture change in the second factor: Stability and change in cultural consensus and residual agreement. Field Methods. 2015;27(1):22–38.
- 67. Romney AK, Batchelder WH, Weller SC. Recent applications of cultural consensus Theory. Am Behav Sci. 1987;31(2):163–77.
- 68. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. pmid:843571
- 69.
Agresti A. Categorical Data Analysis. 3rd ed. Hoboken, NJ: John Wiley & Sons, Inc. 2013.
- 70.
Borgatti SP, Everett MG, Freeman LC. Ucinet 6 for Windows: Software for Social Network Analysis. Harvard, MA: Analytic Technologies. 2002.
- 71. Romney AK. Culture consensus as a statistical model. Curr Anthropol. 1999;40(S):S103–15.
- 72. Henderson NL, Monocello LT, Else RJ, Dressler WW. Modeling culture: a framework. Ethos. 2022;50(2):111–30.
- 73. Boster JS. Exchange of varieties and information between Aguaruna manioc cultivators. Am Anthropol. 1986;88(2):428–36.
- 74. Boster JS, Johnson JC. Form or function: a comparison of expert and novice judgments of similarity among fish. Am Anthropol. 1989;91(4):866–89.