https://orcid.org/0000-0003-4185-0733
Figures
Abstract
Although previous studies of today’s globalised and competitive research landscape have mentioned the research collaborations of CANZUK countries (i.e., Australia, Canada, New Zealand, and the United Kingdom), none have yet studied them in detail. Further, such studies have used different measures of international research collaboration (IRC), resulting in disparate findings. This paper, therefore, analyses the strengths of CANZUK research collaborations, how those collaborations have changed over time, and assesses the effect of three ways of measures on the results (absolute strength, bilateral similarity, and multilateral similarity). We provide a detailed characterisation of the CANZUK research network and its relationships with partner countries, which reveals that the most collaborative CANZUK countries are the UK and Australia, among other findings. We also confirm that many findings differ depending on which measures are used. We offer an explanation of this difference with reference to the nature of the measures (i.e., what they really measure) and make suggestions for suitable measures in future studies depending on their purpose. Finally, we discuss how this study’s findings can be used by research policy makers (in CANZUK and elsewhere) in deciding on research strategy and by researchers in appropriately measuring IRC.
Citation: Nguyen BX, Luczak-Roesch M, Dinneen JD (2024) Do research collaborations age like wine? Absolute and relative measures of CANZUK research partnerships’ strength since the 1950s. PLoS ONE 19(4): e0299319. https://doi.org/10.1371/journal.pone.0299319
Editor: Claudia Noemi González Brambila, Instituto Tecnologico Autonomo de Mexico, MEXICO
Received: January 14, 2022; Accepted: February 7, 2024; Published: April 16, 2024
Copyright: © 2024 Nguyen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Relevant data and codes files are available from the Github database https://github.com/baxuan/IRC-Strength-Measurement
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
International research collaboration (IRC) has become a more developed and important topic in bibliographic research [1, 2]. Determining the patterns of research collaborations between countries is one core area of IRC inquiry [2]. This research is important for understanding the global scholarly landscape, with the potential to shed light on how international actually is and whether there are any imbalances and even inequities among scholars’ involvement, or their ability to be involved, in high-impact research partnerships based on their location.
In this paper we investigate IRC among the so-called CANZUK countries, Canada (CAN), Australia (AUS), New Zealand (NZL), and the United Kingdom (GBR). The countries in this study are referred to by their corresponding three-letter country abbreviations (in ISO-3166 Alpha-3 standard). The explanation of these three-letter country abbreviations is presented in S1 Table.
There are a number of discussions on how tight the pairwise connections between CANZUK countries have been [3, 4], and increasing interest in whether these connections will be tighter or looser after “Brexit” [5]. However, there has been no research studying the research collaboration (RC) of the countries in this cluster as a whole. Nor have there been studies on how these countries, which typically engage in high levels of research activity, are connected to other countries outside of the CANZUK cluster. Thus, an examination of the different ways of measuring CANZUK’s RC strengths is necessary because each way of measuring serves a different purpose [6] and results in a different IRC network map [7].
This paper aims to analyse the strengths of CANZUK research collaborations, and how these collaborations have changed over time. The paper is structured as follows: The ‘Related work’ section introduces a brief description of the fundamentals and related work of the measurement for IRC patterns. The ‘Materials and methods’ section reports the data and methods used in the study. The ‘Results’ section describes the main findings. The ‘Discussion’ section presents the discussions of this paper. The paper concludes by providing insights and recommendations for future research in the ‘Conclusion and future work’ section.
Related work
IRC networks and IRC patterns
Various properties of global collaboration networks have been uncovered in previous studies. Firstly, IRCs have been developing in a preferential attachment process, in which those countries that have already been research cores are continuing to develop more collaborations [8]. Secondly, these networks have been recognised as ‘‘small worlds” in which any two researchers are often connected by a short path of collaborations [9].
However, these networks have been described differently over different periods. In the period 1980–2000, the centre ‐ periphery relationship was examined for the IRC network [7]. Once the IRC research network had expanded with the participation of more nations, this global network also became more interconnected between 1990 and 2000 [8]. Some of the regional networks were considered semi-peripheral regions [10], and a more competitive multi-centric core was observed [11].
The comparison of countries’ scientific profiles has also been used to classify countries having similar IRC patterns side by side. A study by Okubo et al., which examined the publications of eight large scientific fields in the period 1981–1986, revealed a proximity map in which there were many clusters of countries’ scientific profiles [12]. One notable cluster includes Great Britain, which is the country at the centre of this proximity map, and the USA, Canada, Australia, New Zealand and Nigeria. Although the proximity map suggests possible historical effects shared by the countries having similar patterns of countries’ scientific profiles, it does not mean these countries have close research relationships.
Mapping IRC networks and the CANZUK research cluster
Graphical presentations of IRC networks or network ‘maps’ are useful to provide an overview of IRC relationships. Co-authorship maps have been commonly used to present these IRC relationships. At the national level, each node in a co-authorship map represents a collaborative country and each connection represents the RC between a pair of countries. Four co-authorship clusters have been described with a tiny one including Australia and New Zealand, and this tiny cluster connects to the Western European cluster through the Australia-UK connection [13–15]. A co-authorship map describes IRC’s scientific factors through the sizes of its nodes (describing the total numbers of countries’ IRCs) and its connections (describing the numbers of IRCs between collaboration pairs).
Although a separate research cluster including Australia and New Zealand has been mentioned in prior studies, little is known about the RCs among CANZUK countries, and the RCs between the CANZUK members and other countries. Therefore, it is unclear for example, whether the conclusion that the IRC of GBR has been impacted more by historical connections than geographical proximity [16] may be true just for GBR or for the other CANZUK members as well.
IRC measures
IRC studies have used different measurements to analyse the research collaboration patterns across countries. Table 1 shows the representative IRC pattern studies listed by Chen, Zhang, and Fu [2], and the corresponding measurements used in these studies.
There are generally three different approaches to quantify the RCs between countries [7, 13, 14, 23–26]. The first approach is the ‘absolute strength’ measure [13, 27], implemented by counting the bilateral RCs’ observed frequencies. The meaning of the term ‘strength’ used in this context is ‘activity’ of research collaboration [7]. The other approaches are in the group of ‘relative strength’ measurements. To measure the relative strength of scientific linkages in science, the measurements are either size dependent (e.g., AFI which takes the total co-authorship linkage of one country into calculation) or size-independent (e.g., Salton’s measure which normalises the size of total co-authorship linkage of both countries under survey). While the size dependent measurements emphasise on the important research partners, the size-independent measurements demonstrate the links [28] between countries’ IRC strength and their proximities (e.g., cultural proximity or linguistic proximity). In this study, we aimed to explore the relative strength of countries in their global context, which led us to choose size-independent measurements. Within this group of size-independent measurements, two different approaches emerged: bilateral similarity measures, and multilateral similarity measures as follows:
The second approach is bilateral similarity measures [7, 8, 13, 29, 30]. In this approach, the observed bilateral RC numbers are adjusted by the total RC numbers of the two countries involved. There are many ways to implement this normalisation but the most popular ones in the field of scientometrics include the following four measures [31]:
Jaccard index:
and Salton (cosine) index:
where:
Axy: the association strength between two countries, x and y
Ixy: the inclusion index between two countries, x and y
Jxy: the Jaccard index between two countries, x and y
Sxy: the Salton index between two countries, x and y
Cxy: the observed number of their co-authored papers
Cx and Cy is the total number of IRC papers published by countries x and y,
respectively
Theoretically, the association strength measure is classified as a probabilistic similarity measure and the three remaining measures are classified as set-theoretic similarity measures. Probabilistic similarity measures "can be interpreted as measures of the deviation of observed co-occurrence frequencies from expected co-occurrence frequencies under an independence assumption", while set-theoretic similarity measures "can be interpreted as the relative overlap of two sets" [31]. Although probabilistic similarity measures seem to be more appropriate for normalisation purposes, the Jaccard index and the Salton cosine have been the two most popular measures in scientometric research [31].
The third approach is multilateral similarity measures. This approach relates the relative strength of two countries’ bilateral RCs to their collaborative relations with other countries [7, 8, 16, 28]. These relative strengths are calculated by getting the ratio of observed and expected values of bilateral RCs, in which the expected values are obtained with the following formula:
where:
Exy: the expected values of bilateral RCs between two countries, x and y, in comparison to collaborative relations with other countries.
Cx and Cy is the total number of IRC papers published by countries x and y, respectively
T: the total number of IRC papers of all countries.
Consequently, the collaborative networks produced by applying the above three measure approaches represent the symmetric relationships between countries. The reasons are because bilateral RCs between a pair of countries are counted twice, both as an incoming and outgoing edge in collaborative networks (in the ‘absolute strength’ measure), and the denominators in the formulas of these three measures use both the total numbers of IRC papers published by two countries as a way of normalisation (in bilateral similarity measures and multilateral similarity measures).
Two underlying counting methods have been commonly used in the above measures: the whole counting method and the fractional counting method [32]. Of these, the former measures the collaborative activities by assigning a full credit to each unique country having participants in a co-authored publication [12]. The latter measures the collaborative activities by assigning fractional credit to each country based on the proportion of authors it contributes to a given publication [7]. However, the number of collaborative activities alone cannot indicate how strong the RCs of the countries involved are, as measured by the group of ‘relative strength’ measurements. The reason for this is that a number of bilateral RCs considered large (i.e., to small countries) may be just a small RC number to big countries. In order to estimate the ‘propensities’ of RCs, for example, we need to examine the observed/expected ratio, so the countries’ sizes of IRC publications need to be taken into account.
The literature of IRC networks shows that there are research gaps about the CANZUK countries’ RCs, and how different measures can and should be used to reflect these RCs.
Materials and methods
The purpose of the current study is to examine the CANZUK countries’ RCs by answering three RQs:
- RQ1: How have the RCs among CANZUK countries developed over time?
- RQ2: How have the RCs between CANZUK and other countries developed over time?
- RQ3: To what extent do the different measures of RC reveal different IRCs?
In this study, we used bibliographic data to quantify the RCs between countries. We also applied different ways (absolute strength, bilateral and multilateral similarities) to measure the RC strengths and to compare the results. The details are as follows:
Data
Due to budget constraints, we used Microsoft Academic Graph (MAG), a general scholarly bibliographic data set, to examine the historical research collaborations of CANZUK countries. The MAG data set is publicly available to researchers on the Microsoft website. For this study, the entire data set was extracted via bulk download. The data set encompasses publications from the year 1800 to 2018. Since not all publications from the year 2018 are entirely stored in this version of the MAG data set, the year 2018 was excluded from the study. A previous study [33] demonstrated that the mean number of coauthors in publications has only approached around two authors since the 1950s. Consequently, the period 1951–2017 was selected as the range of data for this study. The Fig 1 below shows the time analysis of the number of publications in the MAG data set in this period.
Given that most of IRC network studies have examined the collaborative network maps in the period 1981–2000 [2], we also aimed to compare the CANZUK’s RCs in this period with those in the previous and later periods. Therefore, the data from the years 1951 to 2017 were collected and separated into three periods: 1951–1980, 1981–2000, and 2001–2017. The Table 2 below displays the number and the percentage of publications in this data set, categorised by document type.
The statistical software environment R with its igraph package was used to identify the RCs between countries from bibliographic information of co-authored publications in the data. We applied a method of completing authors’ country affiliation data in MAG records [34] to resolve missing data. Information about scientific factors ‐ numbers of total publications and international co-authored publications ‐ are extracted from this data source to investigate the research collaborations of CANZUK countries.
Methods
We measured IRC in three different ways: absolute strength, bilateral similarity, and multilateral similarity to examine the development of research collaborations relating to CANZUK countries over time.
To measure the absolute strengths of RCs, we calculated the observed frequencies of RCs. We also analysed the distribution statistics of these RCs to estimate their relation patterns during these periods. To further compare the RCs of CANZUK’s countries, we measured the relative strengths of RCs using the following two ways:
To measure the bilateral similarity values of CANZUK’s RCs, we applied the most popular measures in scientometric research: the association strength, the inclusion index, the Jaccard index, and the Salton index. In the latter two measures, bilateral RCs having higher proportions reflect relatively more active connections between the two corresponding countries.
To measure the multilateral similarity of CANZUK’s RCs, the ratios of observed frequencies to expected frequencies of CANZUK’s RCs were calculated. Among them, those values greater than one imply that the corresponding RCs were more active than expected.
The underlying count used in the above measures could be either the whole counting or the fractional counting. The difference between these methods is that the former reflects the research connections between countries while the latter reflects the research productions (i.e., publications in this case) of the collaborative countries. For example, an article co-authored by one researcher from the UK and two researchers from the US will result in different counts. The whole counting method credits one for each country: the UK and the US. Meanwhile, the fractional counting method credits one half for the UK and one half for the US. Previous bibliometric studies made diverse arguments for using one or both of these counting methods [6]. Because a paper measuring the importance of CANZUK countries’ partners [32] showed that minor differences in results were obtained by applying the two different counting methods, we also applied both these two underlying counting methods to examine whether they affected the visualisations of collaborative networks, and compared the two sets of results.
The co-authorship map technique was applied in all these three different ways to visualise the collaborative networks among CANZUK countries, and the collaborative networks between them and other countries. With absolute strength and bilateral similarity, we converted the strength values to the ratios (of the strength values to the median values) to express the lines’ thicknesses. With multilateral similarity, the values were already expressed in ratios (of observed values to expected values) so conversion was unnecessary.
Results
The results were presented in three sub-sections, which reflects the use of measures to evaluate the absolute strengths, bilateral strengths, and multilateral strengths of CANZUK’s RCs, as follows:
Absolute strengths of CANZUK’s RCs
The observed frequencies of CANZUK’s RCs in the three periods show the following main points:
Firstly, CANZUK countries have increased research collaboration relationships over time (with each other or with non-CANZUK countries). This is presented by the increase of collaboration pairs (involving at least one country in the CANZUK group) during the period 1951–2017 (Table 3). The numbers of co-authored publications have also increased, represented by the median values of credits for co-authored publications in the same table. The increase of co-authored publications can be observed through the boxplots in Fig 2, which show that their interquartile ranges have generally moved up over time.
The numbers of co-authored publications have increased during the period 1951–2017.
The numbers of collaboration pairs and credits for co-authored publications between collaboration pairs, calculated by both whole and fractional counting methods, have increased over time.
Secondly, the distributions of RC strengths of CANZUK countries only, comprising six pairings between the four CANZUK countries, typically exhibit a shape with a low and wide peak, and a correspondingly thin tail on the right, as shown in Fig 3. Table 4 gives statistics of these distribution’s shapes. For the RC network of CANZUK countries only, the kurtosis values show that these distributions have a lighter tail than a normal distribution (kurtosis is less than 3). Meanwhile, the skewness values have decreased over time, indicating that these distributions have become more symmetrical.
Histograms of distributions in the three periods of (a) RCs among CANZUK countries, and (b) RCs between CANZUK members and other countries (excluding the CANZUK only pairs). The distributions of (a) have a lighter tail while the distributions of (b) have a heavy tail.
The values show that the distributions of RC strengths among CANZUK countries and the distributions of RC strengths between the CANZUK members and other countries are different.
Thirdly, the distributions of RC strengths between the CANZUK members and other countries, as illustrated in Fig 3 and detailed in Table 4, exhibit a sharp peak and a correspondingly fat tail. A difference is, unlike the distributions of RC relationships of CANZUK countries only having a lighter tail, the RC relationships between the CANZUK members and other countries have a heavier tail than a normal distribution (kurtosis is greater than 3). The heavy tails of distributions of RC relationships between the CANZUK members and other countries, as presented in Fig 3, mean that these distributions are outlier-prone.
Fourthly, research connections among CANZUK countries have more RCs than the median values of all CANZUK’s RCs, as shown in Fig 4 and Tables 5–7. The images in Fig 4 show the ‘absolute strength’ of RC relationships among CANZUK countries and the top three bilateral research relations (i.e., having most active RCs) between CANZUK countries and outside countries (described as top three from now on). The relationships among CANZUK countries are presented as solid lines in blue, while the relationships between CANZUK countries and other countries are presented by dashed lines in green. The thicknesses of the lines in Fig 4 represent the relative strength of research connections to their median values.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between individual members of the CANZUK group and other countries.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between individual members of the CANZUK group and other countries.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between the CANZUK members and other countries.
Tables 5–7 below show a comparison of RCs’ strengths among CANZUK countries, and the top three bilateral research relations between CANZUK and other countries across the three periods. Because the numbers of RCs have increased over time (Table 5), we did not normalise the absolute values of RCs’ strengths by year for comparison across the three periods. Instead, we calculated the ratio of each relationship’s strength to the median strength of all relationships involving CANZUK countries in each period. These ratios were then used for comparison, as presented in Tables 5–7. These tables show that the ratios have increased over the three periods 1951–1980, 1981–2000, and 2001–2017. In other words, the strengths of RC relationships among CANZUK countries have become relatively more active. This indicates that the CANZUK countries increased their collaborations at a much higher speed than the collaborations of the rest of the world.
Fifthly, GBR has remained the top collaborator within the CANZUK group. Meanwhile, the USA was in the top research collaborations with the CANZUK group throughout the whole period (1951–1980). Only DEU (in the period 1981–2017) was also one in the top three of CANZUK’s important partners. These countries are also ranked by their RC weights, as represented in Tables 8–10. These tables show the RC weights of CANZUK countries and countries involved in the top three RCs with CANZUK countries. The value for each CANZUK country is the sum of all weights of RCs between that country and other CANZUK countries while the value of a non-CANZUK country is the sum of all weights of RCs between that country and a CANZUK country.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between the CANZUK members and other countries.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between the CANZUK members and other countries.
Two groups of RCs were examined: RCs among CANZUK countries and the top three RCs between the CANZUK members and other countries.
Lastly, the two methods (whole counting and fractional counting) give dissimilar RC strength rankings. For example, the whole counting method shows that the USA and DEU are the two top partners of the CANZUK group in the period 1981–2017 whereas the fractional counting method presents the USA as the only top partner during this period. The difference occurs because DEU had more collaborations with CANZUK countries and therefore was credited with more whole counts. Meanwhile, the USA was credited with more collaboration proportions because their co-authored publications counted fewer total partners.
Bilateral similarity of CANZUK’s RCs
From analysing the absolute strengths of CANZUK’s RCs as in the above section, we can observe that the numbers of co-authored publications have increased over time. However, this apparent growth may not be due to any change in the nature of collaborations between these countries but simply in general growth in these countries. For example, several factors could explain this, including population growth and the increase in the number of scientists in these countries, among others. Therefore, we applied a normalisation of these numbers. The bilateral similarity approach normalises these numbers using the total number of IRC papers published by the two countries in a collaboration pair.
Figs 5 and 6 present the co-authorship maps of RC among CANZUK countries, calculated using the whole counting method and the fractional counting method, respectively. Each of these figures visualises the comparison between the results calculated by four applied measures: association strength, inclusion index, Jaccard index, and Salton index. Examining each figure, the patterns of co-authorship maps produced by the Jaccard index and by the Salton index are quite similar, yet distinct from the patterns of co-authorship maps generated by the association strength and inclusion index.
Upon comparing Figs 5 and 6, it becomes evident that the resulting networks portrayed in these two figures are similar when generated using the same measures, except for the ‘association strength’ measure. The utilisation of the association strength measure, combined with either the whole counting method or the fractional counting method, generates slightly different patterns in Figs 4 and 5. The image produced using the whole counting method in this case shows that the five most important collaborations with outside countries are those between the NZL and COK, FJI, LBR, SLB, and TON. On the other hand, the image produced by implementing the fractional counting method shows only three top relations, namely between NZL and LBR, TON, and VUT. Additionally, the image produced using the fractional counting method highlights that AUS and NZL exhibit a stronger collaboration compared to other collaborations among CANZUK countries during the period 2001–2017. However, the image produced by implementing the whole counting method does not demonstrate this phenomenon.
It is also worth noting that there are some similarities in the images generated by using measures to evaluate the absolute strengths, bilateral strengths, and multilateral strengths of CANZUK’s RCs. Specifically, the association strength measure results in images that are similar to those from the multilateral similarity measure (as presented in the following section). Meanwhile, the Jaccard index and Salton index produce images similar to those from the ‘absolute strength’ measure. For example, one notable finding is that the USA has consistently remained a top partner in the RCs with two countries in the CANZUK group ‐ CAN and GBR. These findings can be observed when comparing the images resulting from using the ‘absolute strength’ measure (Fig 4), and those resulting from using the Jaccard index and Salton index (Figs 5 and 6).
However, there are some differences between the results obtained from either the Jaccard index, the Salton index, or the ‘absolute strength’ measure. The most notable finding from the two former measures is that CAN was the CANZUK country that the USA had the strongest research relationship with. This observation is different from the findings obtained from the evaluation of the CANZUK’s RCs’ absolute strengths, which show that the USA had the strongest research relationship with GBR.
The two measures calculating the bilateral similarity of CANZUK’s RC, the Salton index and the Jaccard index, also reflect some slightly different details in the collaboration maps. For the Salton index, the USA and DEU were the two countries that had the three strongest research relationships with CANZUK countries in the period 2001–2017. By contrast, the Jaccard index shows the USA was the only country that had the three strongest research relationships with CANZUK countries. The Jaccard index, therefore, reflects similar results to those obtained from the absolute strengths of CANZUK’s RC with fractional counting, while the Salton index shows similar results with the absolute strengths using the whole counting.
Table 11 shows the values of the Jaccard index and the Salton index of CANZUK’s RCs. In this table, the higher the values are, the more active the RCs are when the sizes of IRC publications of the two relevant countries are applied to normalise these strength values.
The values show that bilateral similarity has decreased over time, except for the case of the AUS-CAN research collaboration from the period 1981–2000 to the period 20012017 (calculated by the Jaccard index), and the GBR-NZL research collaboration from the period 1981–2000 to the period 2001–2017 (calculated by both the Salton index and the Jaccard index).
Multilateral similarity of CANZUK’s RCs
Table 12 shows the comparison of CANZUK’s RCs in the three periods, calculated by taking all the countries’ research sizes to normalise the RCs’ strength values. Those values greater than 1 mean that the corresponding research connections have more RCs than expected. Those values less than 1 mean that the corresponding research connections have fewer RCs than expected. Fig 7 shows the collaboration maps in these periods. This figure shows that the RCs between NZL and some other countries are the mainstream pattern.
The values show that multilateral similarity has decreased over time.
Fig 7 reveals noticeable distinctions between the collaboration maps derived from CANZUK’s RCs using the multilateral similarity measure, as compared to both the collaboration maps based on the absolute strength measure and those based on the bilateral similarity measure of CANZUK’s RCs. In this figure, the thick lines describe connections among CANZUK countries having more observed RCs than expected while the thin lines describe connections among CANZUK countries having fewer observed RCs than expected. In terms of expectation, only the relationship between AUS and NZL has had more RCs than expected. Fig 7 also shows that the top collaborators with CANZUK countries, in terms of expectation, have changed dramatically over time. In the period 1951–1980, NZL had top research connections with the Cook Islands, Fiji, Liberia, Tonga, Vanuatu, and the Solomon Islands; or just Liberia, Tonga, and Vanuatu, depending on whether the whole counting method or the fractional counting method was applied in calculating multilateral similarities. In the period 1981–2000, Samoa, Tokelau, and Tonga were the top collaborators with the CANZUK countries through RCs with NZL. In the period 2001–2017, NZL had top research connections with Samoa, American Samoa, and Niue. We also carried out chi-square goodness-of-fit tests to compare the observed distribution to the expected distribution of RCs. Table 13 shows that only the chi-square tests’ p-value of the test using fractional counting for the period 2001–2017 is higher than 0.05, which means that generally there is a significant difference between the observed and the expected RCs at the 5% significance level. In other words, the thicknesses of the lines in Fig 7 are meaningful because their corresponding ratios (of observed values to expected values) are significantly different from 1.
In summary, the various measurement approaches for IRC produce different findings. Table 14 provides an overview of the measures employed and their corresponding results.
Discussion
The goal of our investigation is to examine the research collaborations of CANZUK countries. Here we discuss the findings of this study. The discussion is structured around the research questions and related to previous research. Additionally, this section ends with a summary detailing how the results support the paper’s objectives.
RQ1: How have the RCs among CANZUK countries developed over time?
The RCs among CANZUK countries have been a symmetrical network, meaning it is evolving towards complete connectedness and equal distribution. Since this network symmetry relates to system robustness [35], we infer that the RCs among CANZUK countries have been in a more sustainable network, in which every country has diversified and more balanced relationships with other countries in the group. A sustainable network state is a preferred state for “physical (constructed and natural) systems and human communities (social and institutional)” in order to “possess the capacity to survive, cope, recover, learn and transform from disturbances’’ [36], or to "be able to survive and function under extreme stress" [37]. However, this development is still below what is possible because the numbers of IRC publications between the CANZUK countries are relatively low in comparison with each country’s total IRC publications. The only exceptional case, in terms of expectation, is the strong collaboration between AUS and NZL. This strong collaboration is consistent with another study’s findings for the period 1999–2005 [4].
RQ2: How have the RCs between CANZUK and other countries developed over time?
The RCs between CANZUK and other countries have also developed during the years under investigation, but the top collaborating partners have been mainly determined by the top collaborators [32] within the CANZUK group. The more active RCs the UK and Canada, and the USA reflect the traditionally close economic and political links between these nations. In general, the strong research partnerships between the CANZUK group and the USA have been explained by their historical relationships [7, 16], and as a result of “The Technical Cooperation Programme" comprising of the USA, UK, Canada, Australia and New Zealand [3, 38].
By contrast, the other positions of the top partners in each period reflect the RC policy in CANZUK countries during these times. Depending on the measure used, DEU might be considered having one of the three top RCs with the CANZUK. Since the former East German states joined the Federal Republic of Germany in 1990, Germany gradually strengthened its research capacity and became an important research partner (after the USA) with CANZUK countries, using the whole counting method by Salton index. Although there has been an increase in research collaborations between China and the most developed countries in the world, including England and Canada [24], as a result of China’s reform policy, China was not in the three top research partners with CANZUK. Previous studies have separately mentioned the strong research relationships between some of the CANZUK countries and Germany, and the USA [8, 16]. For example, the study by Zitt et al. [16] describes the research collaboration among five countries having large scientific research sectors, namely France, Germany, Japan, the United Kingdom, and the United States. This study includes the descriptions about the research collaborations between one of the CANZUK countries, the United Kingdom, with other countries, using the 3-year-average values of the two separate periods, 1996 (1995–1997) and 1986 (1985–1987). By contrast, our study examines and presents the RCs between all CANZUK countries and other countries throughout the period 1951–2017, which covers the time coverage in a previous study [32]. In other words, our study gives an overall picture of the development of RCs between CANZUK and other countries. Therefore, our study provides beneficial values for researchers and policy makers who want to have a systematic overview of the IRCs between the CANZUK group and other countries. A complete picture of RCs between countries is essential for identifying strategic collaborations [39] so that countries can raise their innovation competitiveness [40]. Our study also finds that the actual collaborating outcomes may be interpreted differently, depending on the approach applied. For example, the multilateral similarity measure approach considers the research relations with all countries’ RCs. In this approach, different periods see different CANZUK countries performing RC with others over what is expected. These differences were described in the next research question.
RQ3: To what extent do the different measures of RC reveal different IRCs?
The measures applied in this study show different patterns of RC development and different patterns of RC rankings. Figs 4–7 show different results obtained by different approaches to measuring the RCs’ strengths: by absolute strength measure, by bilateral similarity measures and by multilateral similarity measure. On the one hand, the absolute strength measure shows that research relationships among CANZUK countries have developed over time. In detail, this measure shows that the research relationships among CANZUK countries have had more RCs in average (compared to the median value) and these research relationships have also increased their relatively normalised strengths. On the other hand, the bilateral similarity measures and the multilateral similarity measure show that the relative strengths of research relationships among CANZUK countries have developed less over time.
However, the ranks of countries having RCs with the CANZUK countries by these measures are similar: the USA has consistently been the most popular country for CANZUK countries to partner with, and GBR is the most collaborating country in the CANZUK group.
It is worth noting that there are differences in patterns of RC development and rankings given by these above measures, and these differences can be explained by the nature of different ways of calculation. For the absolute strength measure, the Jaccard index, and the Salton index: the numbers of RCs between each pair of countries are not normalised or are normalised by the ‘average’ size of the two corresponding countries’ IRC publications. For the second group (the association strength measure and the multilateral similarity measure): the numbers of RCs between each pair of countries are normalised using the product of the two corresponding countries’ amounts of IRC publications. Therefore, the association strength measure and the multilateral similarity measure tend to give higher scores for the research relationships between two countries having more gaps in the amounts of IRC publications (e.g., one country has a low number of IRC publications while the other country has a high number of IRC publications), which can be seen in the cases of NZL in comparison with its research partners (Figs 5 and 7).
Previous research has suggested that the impacts of different counting methods should be considered in IRC measurement studies. For each measure in this study, the two underlying counting methods applied also lead to slight differences in particular rankings. An example is the case of the absolute RC strength measure presented in Fig 4. In this figure, the ranking of the top collaborating countries with the CANZUK group varies for each of the examined periods, depending on whether the whole counting method or fractional counting method is used. The three top strongest RCs feature different CANZUK partners (the USA only, or the USA and DEU).
The differences in rankings given by the whole counting method and the fractional counting method can also be explained by the nature of their calculation. The former is more suitable for studies focusing on the number of research relationships between co-authors, while the latter is more suitable for studies focusing on the number of publications credited to the countries involved in the research collaboration. These differences in the results obtained by applying these two methods are notable because they tell different stories about the CANZUK RCs. For example, the whole counting method shows that the CANZUK group still has more active RCs with their traditional European partner, DEU, while the fractional counting method shows that the USA has surpassed DEU to become the only top research partner of CANZUK countries in the period 1981–2017.
The substantial differences observed in the rankings of the top collaborators with CANZUK countries are consistent with findings in previous literature. For example, research studying the problem of multiple authorship [41] shows that the difference between whole counting and fractional counting is small when measuring low frequency collaborators, but that the difference becomes substantial when measuring high frequency collaborators. The difference is caused because the whole counting method amplifies the credits of countries having more RCs in multi-authored publications. The fractional counting reduces this bias by dividing each publication by the number of co-authors [41]. As both methods have biases [29], researchers prefer to use either the whole counting [12] or the fractional counting method [16, 42]. This study measures the collaborative strength of CANZUK countries so both underlying counting methods are useful to examine two different aspects of CANZUK’s collaborative strengths: the former describes the relationship-based strengths while the latter describes the production-based strengths.
The other findings suggest two recommendations for choosing suitable measures of research collaboration strengths. First, the choice of measures should depend on the purpose of measurement. The absolute strength is suitable for comparing the numbers of RCs between countries. The bilateral similarity measures are suitable for comparing the relative strengths of research relationships between pairs of countries, in relation to the two corresponding countries’ sizes of IRC publications. The multilateral similarity measure is preferable when examining whether the ‘observed’ strengths of research relationships between pairs of countries meet their ‘expected’ values. Second, the Jaccard index and the Salton index can be considered for alternative use if the purpose of IRC measurement is to compare the relative strengths of research relationships. For example, a study applying the Salton index measure about international research collaboration in the period 1981–1985 [13] showed that the USA had strong collaboration links with the UK and Canada. This study would have had significantly different results if the authors had used a different measure, such as association strength or inclusion index (as illustrated in Figs 5 and 6).
In summary, we aimed to analyse the evolution of CANZUK research collaborations over time and assess their strength using various measures. To achieve this, we examined statistical methods related to research collaborations and employed three distinct measurement approaches: ’absolute strength’, ’bilateral similarity’, and ’multilateral similarity’. Our analysis of descriptive statistics revealed that CANZUK countries have developed notably, both in the total number of collaboration pairs and in the median values of co-authored publications among CANZUK countries when compared to other nations, forming a more balanced research network. However, the growth of these research collaborations was not as much as expected. Differences between the observed and expected values of CANZUK’s research collaborations were supported by chi-square goodness-of-fit tests. The results suggest a growing trend in research collaboration relationships among CANZUK countries over time, with the potential for further development.
Furthermore, the three measurement approaches consistently identified the USA as the top outside collaborator of the CANZUK countries. Pivotal collaborators within and beyond the CANZUK group can be explained by historical, economic, and political influences.
Moreover, the application of the three measurement approaches to assess collaboration strengths revealed diverse patterns and rankings of countries involved in research collaborations. Despite variations, notable similarities emerged in the results. Specifically, the association strength measure of ’bilateral similarity’ produced outcomes similar to those obtained through the ’multilateral similarity’ measure. Conversely, the Jaccard index and Salton index of ’bilateral similarity’ exhibited similarities with the ’absolute strength’ measure.
Conclusion and future work
In conclusion, the present study aimed to find out how the RCs of CANZUK countries have developed over time. Our findings suggest that these collaborations get better over time, demonstrating a balanced growth in bilateral partnerships within the CANZUK group, as well as an increase in active collaborations with other nations. In addition, different measures gave notably different results when they were used to examine the strengths of RCs between countries. The CANZUK’s collaboration map and the ranking lists of top collaborating countries with the CANZUK group were depicted differently depending upon the measures used. These differences might create conflicting conclusions in RC studies. Therefore, the choice of suitable measures should depend on the purpose of the RC studies.
There are limitations in the present study and opportunities to explore further. Firstly, this study chose the MAG data source to examine the research collaborations of CANZUK countries and the results from this choice may be different than those obtained if other data sources were used. A study about the effects of data set choice on measuring IRC has shown that different data sources give slightly different outcomes of IRC measurement [43]. Therefore, finding which data source is the most suited for IRC measurement and its corresponding statistics of CANZUK’s research collaborations should be another focus of IRC studies. Secondly, this study used co-authored publications as the common outcomes of research collaborations. However, there are various types of outcomes beyond joint research publications: patents, joint research grants, etc. [44], and different rewards [45] of the collaborations: acknowledgements in PhD theses, research journals (articles, editorials, reviews, etc.) and books. The use of only co-authored publications could have overlooked the cases where there was no co-authored publication; or reflected just a part of the research collaboration between countries. These other potential outcomes, together with co-authored publications, should be examined in future work to reveal a more accurate picture of the CANZUK RC network.
Despite the limitations, the present study is important because it provides insights into the RCs between CANZUK countries. We are explorers in studying the research collaboration of the CANZUK countries as a group and our exploration has shown that their shared histories of language, cultural tradition, and economic developments can encourage their research connections although they do not have close geographical proximity. Therefore, the depth and quality of these partnerships undoubtedly improve with time, much like the refinement of wine with age. Our study is also a pioneering work in applying and comparing different measures of IRC assessment. For research policy, the study shows that the RCs among CANZUK countries have been in a balanced network but these RCs perhaps remain below what is possible, in relation to their numbers of IRC publications. In other words, the CANZUK countries could have further strengthened their research collaborations. These strong research collaborations are increasingly important for the CANZUK countries, especially as the UK has left the European Union (i.e., needs to review its research partnerships) and Australia is looking to strengthen its collaborations with long-established allies via agreements such as the trilateral security pact (AUKUS) with the United Kingdom and the United States [46]. The circumstances mentioned above can encourage policy makers to support further research collaborations among CANZUK countries.
Supporting information
S1 Table. Explanation for the list of the current officially assigned ISO 3166–1 alpha-3 codes and their corresponding English country (or dependent territory) names in the present study.
https://doi.org/10.1371/journal.pone.0299319.s001
(DOCX)
References
- 1. Luukkonen T, Persson O, Sivertsen G. Understanding patterns of international scientific collaboration. Science, Technology, & Human Values. 1992 Jan;17(1):101–26.
- 2. Chen K, Zhang Y, Fu X. International research collaboration: An emerging domain of innovation studies?. Research Policy. 2019 Feb 1;48(1):149–68.
- 3. Carter G, Pearson GS. North Atlantic chemical and biological research collaboration: 1916–1995. The Journal of Strategic Studies. 1996 Mar 1;19(1):74–103.
- 4. Benckendorff P. Exploring the limits of tourism research collaboration: A social network analysis of co-authorship patterns in Australian and New Zealand tourism research. CAUTHE 2010: Tourism and Hospitality: Challenge the Limits. 2010:151.
- 5. Gamble A. The Brexit negotiations and the Anglosphere. The Political Quarterly. 2021 Jan;92(1):108–12.
- 6. Gauffriau M. A categorization of arguments for counting methods for publication and citation indicators. Journal of Informetrics. 2017 Aug 1;11(3):672–84.
- 7. Luukkonen T, Tijssen R, Persson O, Sivertsen G. The measurement of international scientific collaboration. Scientometrics. 1993 Sep 26;28(1):15–36.
- 8. Wagner CS, Leydesdorff L. Mapping the network of global science: comparing international co-authorships from 1990 to 2000. International journal of Technology and Globalisation. 2005 Jan 1;1(2):185–208.
- 9.
Newman ME. The structure of scientific collaboration networks. InThe Structure and Dynamics of Networks 2011 Oct 23 (pp. 221–226). Princeton University Press.
- 10. Schubert T, Sooryamoorthy R. Can the centre–periphery model explain patterns of international scientific collaboration among threshold and industrialised countries? The case of South Africa and Germany. Scientometrics. 2010 Apr 1;83(1):181–203.
- 11. Gui Q, Liu C, Du D. Globalization of science and international scientific collaboration: A network perspective. Geoforum. 2019 Oct 1;105:1–2.
- 12. Okubo Y, Miquel J, Frigoletto L, Doré J. Structure of international collaboration in science: Typology of countries through multivariate techniques using a link indicator. Scientometrics. 1992 Oct 26;25(2):321–51.
- 13. Schubert A, Braun T. International collaboration in the sciences 1981–1985. Scientometrics. 1990 Jul 1;19(1–2):3–10.
- 14. Glänzel W. National characteristics in international scientific co-authorship relations. Scientometrics. 2001 Apr 26;51(1):69–115.
- 15.
Glänzel W, Schubert A. Analysing scientific networks through co-authorship. InHandbook of quantitative science and technology research 2004 (pp. 257–276). Springer, Dordrecht.
- 16. Zitt M, Bassecoulard E, Okubo Y. Shadows of the past in international cooperation: Collaboration profiles of the top five producers of science. Scientometrics. 2000 Mar 26;47(3):627–57.
- 17. Glanzel W, Winterhager M. International collaboration of three East European countries with Germany in the sciences, 1980–1989. Scientometrics. 1992;25(2):219–27.
- 18. Nagpaul P. Transnational linkages of Indian science: a structural analysis. Scientometrics. 1999 Sep 1;46(1):109–40.
- 19. Garg KC, Padhi P. A study of collaboration in laser science and technology. Scientometrics. 2001 Jun;51(2):415–27.
- 20. Gazni A, Sugimoto CR, Didegah F. Mapping world scientific collaboration: Authors, institutions, and countries. Journal of the American Society for Information Science and Technology. 2012 Feb;63(2):323–35.
- 21. Tan HX, Ujum EA, Choong KF, Ratnavelu K. Impact analysis of domestic and international research collaborations: a Malaysian case study. Scientometrics. 2015 Jan;102(1):885–904.
- 22. Vakilian M, Majlis BY, Mousavi M. A bibliometric analysis of lab-on-a-chip research from 2001 to 2013. Scientometrics. 2015 Nov;105(2):789–804.
- 23. Anuradha K, Urs S. Bibliometric indicators of Indian research collaboration patterns: A correspondence analysis. Scientometrics. 2007 May 1;71(2):179–89.
- 24. He T. International scientific collaboration of China with the G7 countries. Scientometrics. 2009 Sep 1;80(3):571–82.
- 25. Finardi U, Buratti A. Scientific collaboration framework of BRICS countries: an analysis of international coauthorship. Scientometrics. 2016 Oct;109(1):433–46.
- 26. Hou L, Pan Y, Zhu JJ. Impact of scientific, economic, geopolitical, and cultural factors on international research collaboration. Journal of Informetrics. 2021 Aug 1;15(3):101194.
- 27. Sidone OJ, Haddad EA, Mena‐Chalco JP. Scholarly publication and collaboration in B razil: The role of geography. Journal of the Association for Information Science and Technology. 2017 Jan;68(1):243–58.
- 28. Chinchilla-Rodríguez Z, Bu Y, Robinson-Garcia N, Costas R, Sugimoto CR. Travel bans and scientific mobility: utility of asymmetry and affinity indexes to inform science policy. Scientometrics. 2018 Jul;116(1):569–90
- 29. Chinchilla-Rodríguez Z, Bu Y, Robinson-García N, Sugimoto CR. An empirical review of the different variants of the probabilistic affinity index as applied to scientific collaboration. Scientometrics. 2021 Jan 3:1–21.
- 30. Pohl H. Collaboration with countries with rapidly growing research: supporting proactive development of international research collaboration. Scientometrics. 2020 Jan;122(1):287–307.
- 31. Eck NJ, Waltman L. How to normalize cooccurrence data? An analysis of some well‐known similarity measures. Journal of the American society for information science and technology. 2009 Aug;60(8):1635–51.
- 32. Nguyen BX, Dinneen JD, Luczak-Roesch M. Closed Shop or Collaborative Hub? An Analysis of the Partners’ Importance in CANZUK Countries’ Research Collaborations. Frontiers in research metrics and analytics. 2022;7. pmid:35958294
- 33. Fortunato S, Bergstrom CT, Börner K, Evans JA, Helbing D, Milojević S, et al. Science of science. Science. 2018 Mar 2;359(6379):eaao0185. pmid:29496846
- 34. Nguyen BX, Dinneen JD, Luczak-Roesch M. A Novel Method for Resolving and Completing Authors’ Country Affiliation Data in Bibliographic Records. Journal of Data and Information Science. 2020 Jul 20;5(3):97–11
- 35. Sánchez-García RJ. Exploiting symmetry in network analysis. Communications Physics. 2020 May 15;3(1):1–5.
- 36. Simonovic SP, Peck A. Dynamic resilience to climate change caused natural disasters in coastal megacities quantification framework. British Journal of Environment and Climate Change. 2013 Jul 1;3(3):378.
- 37. Godschalk DR. Urban hazard mitigation: creating resilient cities. Natural hazards review. 2003 Aug;4(3):136–43.
- 38. Cumming AS. Results from Research Collaboration–A Review over 20 Years. Propellants, Explosives, Pyrotechnics: An International Journal Dealing with Scientific and Technological Aspects of Energetic Materials. 2009 Jun;34(3):187–93.
- 39. Xiang D, Li H. Analyzing international scientific collaboration pattern for China by using ESI database. In2012 IEEE International Conference on Industrial Engineering and Engineering Management 2012 Dec 10 (pp. 1386–1390). IEEE.
- 40. Hwang K. International collaboration in multilayered center-periphery in the globalization of science and technology. Science, Technology, & Human Values. 2008 Jan;33(1):101–33.
- 41. Lindsey D. Productio and citation measures in the sociology of science: The problem of multiple authorship. Social Studies of Science. 1980 May;10(2):145–62.
- 42. Leclerc M, Gagné J. International scientific cooperation: The continentalization of science. Scientometrics. 1994 Nov 26;31(3):261–92.
- 43. Nguyen BX, Luczak-Roesch M, Dinneen JD. Exploring the Effects of Data Set Choice on Measuring International Research Collaboration: an Example Using the ACM Digital Library and Microsoft Academic Graph. arXiv preprint arXiv:1905.12834. 2019 May 30.
- 44. Yuan L, Hao Y, Li M, Bao C, Li J, Wu D. Who are the international research collaboration partners for China? A novel data perspective based on NSFC grants. Scientometrics. 2018 Jul;116(1):401–22.
- 45. Laudel G. What do we measure by co-authorships?. Research evaluation. 2002 Apr 1;11(1):3–15.
- 46. Leece D. AUKUS: Its strategic and diplomatic implications. United Service. 2021 Dec 1;72(4):3–4.