Problem formulation

The key point of this paper is to predict the formation of network connections. For this purpose, it is important to combine factors that measure different aspects of the nodes into a single data structure. The issue can be articulated as follows: given a node pair i and j, where i and j may be (incumbents) or may not be (newcomers) part of the current network, we aim at predicting the probability of this pair to generate a relationship in the next future.

Suppose G(N, L) to be an undirected graph, with N a set of nodes (e.g. the universities funded under Horizon 2020 between 2014 and 2016 in the three ERC domains), and L representing the link between nodes. Loop and multiple links among nodes are not considered in this analysis. Our aim is to predict the link probability of a given node with respect to all the other nodes of the graph. Given a snapshot of the network at time t, we seek to predict what links are likely to be created (or, possibly, kept) in a future time t′. Fig 1 shows the observed links at time t, and the link prediction at time t′, considering a new node F that did not belong to the network at time t.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Representation of two graphs: Observed links at time t, and missing link prediction at time t′.

https://doi.org/10.1371/journal.pone.0290018.g001

By means of popular machine learning algorithms, for each node in the network, we determine all the possible link probabilities with the other nodes. In the simplified example of the figure, node F has zero link probabilities with all the other nodes, except for node B and D. The link with node B has a 70% likelihood to occur, against a 30% of the link with node D. It is thus likelier that university F will collaborate with university B in a new round of the research funding programme herein analyzed. Of course, it is relevant to evaluate to what extent these probabilities are accurately estimated. This is the aim of the first part of our study (first question), where we set out the out-of-sample performance accuracy of such predictions using several machine learning predictive algorithms. The accuracy is clearly conditional on the features employed. As already said, in fact, the features can be endogenous to the nodes (e.g., Jaccard coefficient and Betweenness centrality), or exogenous to the nodes (e.g., the gross domestic product of the region a node belongs to).

The distinction between endogenous and exogenous features is relevant as it entails a prediction trade-off we aim at exploring in this paper (second question): indeed, if we use both endogenous and exogenous features, we are unable to predict link probabilities for the so-called newcomers in the programme, as these entities have no (past) information on network endogenous measures, as they have never been part of the extant network. In this case, we are obliged to rely only on those entities that are already part of the network, as for them we know both endogenous and exogenous features. This is a limitation, but it lends larger predictive power, as long as the endogenous features are highly correlated with link prediction (as a large empirical evidence supports). In contrast, relying only on exogenous features allow us for estimating link probabilities also for the newcomers, a task that may be of great interest for understanding future new configurations of the network. However, as in this case we are excluding from the analysis features possibly highly conducive to predict new links, the statistical accuracy of the estimated probabilities could be poorer, undermining the forecasting quality of the network evolution. Providing a measure of the accuracy reduction produced by excluding the endogenous features is one of the contribution of this study.

Finally, this paper provides a more in-depth understanding of the role played by each single feature in steering the probability of setting-up a link (third question). We carry out this analysis by aggregating over all the learners employed (“super-learning”) the average partial effect (APE) of each single feature. For a single feature x_j, the APE is the derivative of the probability to lie a link with respect to the feature x_j itself, with all the other features kept fixed at their sample mean. As machine learning algorithms are highly nonparametric, this derivative is a highly nonlinear function of the level of the feature; exploring APE’s shape sheds light on the relative importance of each feature in driving link probability. Interestingly, beside derivatives, we also explore the pattern of the elasticities to catch the percentage change activated by a one percentage change in every considered feature.

Data preprocessing and step-wise implementation

This paper presents a set of machine learning algorithms for predicting network links. Here, we discuss the different steps we have undertaken for the data preprocessing. Originally, we had a bipartite network (university by project), and the variables at university level. Then, we transformed the network into a one-mode format. The one-mode projection means a network containing all universities’ pairs. According to the ERC panels, we assigned each project to one the following three research domains: Social Sciences and Humanities (SSH), Physical Sciences and Engineering (PE) and Life Sciences (LS). Each project is assigned to a domain based on its topic identifier, keywords, and abstract content. Each project has a topic identifier that allows it to be assigned to an ERC domain. When the topic identifier for the project was ambiguous (no direct association), the project was assigned to a domain based on the content of its abstract. A small number of projects (just over 2% of the total) is excluded from the study because explicitly described as multidisciplinary, thus not having a clear attribution to a research domain. Because of their primarily individual nature, we also exclude Marie Curie and European Research Council projects. To prepare the dataset for running the machine learning algorithms, we separate the target variable and the features, with the features transformed into normally standardized variables. We only have one training dataset, and we measure learners’ goodness-of-fit via a 10-fold cross-validation out-of-sample accuracy. As accuracy measure, we employ the complement of the error rate, that is, the out-of-sample proportion of correct matches found between the actual and the predicted labels. (5) where: N is the number of universities’ pairs; y_i is the link binary variable for pair i (taking value 1 if a link exists, and 0 otherwise); is a generic learner (a classifier, in our case); X_i is a vector of p features (either exogenous or endogenous); and I[⋅] is an indicator variable taking value 1 if the statement into the squared parenthesis is true, and 0 otherwise. Eq (5) can be estimated both in-sample (training accuracy), or out-of-sample (testing accuracy). As usual in machine learning, testing accuracy will be retained as our reference goodness-of-fit, being training accuracy plagued by overfitting).

Also, to take into account the large number of zeros within our link target variable, we carried out random under-sampling.

For carrying out our analysis along the line mentioned above, we proceed according to the following road-map:

1. We start using the EUPRO dataset (a dataset providing information on R&D projects, participants and resulting networks of the EU FPs) to build the R&D project network from 2014 (when the Horizon 2020 programme has started) to 2016. In our network, the node is the university, and the link indicates a collaboration on a joint project within the Horizon 2020 programme.
2. We classify projects according to the ERC panels, by assigning each project to a specific research domain: Social Sciences and Humanities, Physical Sciences and Engineering, and Life Sciences. The assignment is based on project keywords and on the content of the project’s abstract. When a project is multidisciplinary, with no clear designation of its domain, we exclude it from the analysis. In this way we are able to capture the specificity of each ERC domain. Given their larger specificity, we do not consider ERC grants and Marie Curie projects. To circumscribe our analysis to more homogeneous partnerships, we focus on the three ERC domains separately.
3. We use the RISIS-ETER (database on European Higher Education Institutions) database to extract university level variables and the geographical distance for each couple of universities, and combine EUPRO and RISIS-ETER information using the same node ID.
4. We use a proxy for university size based on the number of students belonging to a specific ERC domain, and compute the Jaccard coefficient, the Katz score and Closeness and Betweenness centrality for every node in every year.
5. We apply several supervised machine learning methods for predicting universities’ link probabilities. As our link target outcome is a binary factor variable (taking value 1 if there is a link, and 0 otherwise), we run several binary classification experiments.
6. We compare the performance of the proposed learning algorithms by jointly considering—for every learner—the average out-of-sample (or “test”) accuracy, and the standard deviation obtained by a 10-fold cross-validation resampling procedure.
7. We further compute the accuracy achieved by comparing two specifications of our predictive model, one embedding both exogenous and endogenous features, and one considering only exogenous features. We thus calculate the accuracy gap.
8. For each learner, we then estimate the average partial effects (APE) function, i.e. the derivative of the conditional probability function with respect to one feature, with all the others held fixed at their sample mean.
9. We aggregate all the derivatives obtained in the previous step by averaging over them, thus obtaining a super-learning derivative estimate that we plot, compare, and comment for every feature.
10. As a derivative measures, by definition, infinitesimal changes, we also calculate elasticities to assess the percentage change of link probability induced by a given percentage change in the considered feature.

The features employed for link prediction are:

• EXOGENOUS FEATURES
  • Regional gross domestic product (PPS per inhabitant), measured at regional level from 2014 to 2016. Universities located in regions with a higher gross domestic product (GDP) are likely to hold the necessary capacities and resources to acquire public funding for collaborative projects [40].
  • Core funding, indicating the overall government funding available for a university. It consists mainly of basic government allocation. It can be considered both as a magnitude of financial inputs in knowledge production, and as a “boost” to push universities to increasingly raise funding (e.g., by participating to competitive projects) [41]. It is thus a lever to help generating additional (external) funding.
  • Citation score, measured by the “mean normalized citation score”, is the average number of citations of a university’s publications, normalized by field differences and publication year. This variable is a proxy of university reputation. Citations have been widely used in the scientific literature to capture knowledge outputs [see, for example, 42].
  • Number of students by ERC domain, considered as a proxy of university size re-scaled within the three ERC domains.
  • Inverse of the distance, considered as the Euclidean distance in terms of kilometers between two uniersities. We used the inverse of geographical distance as a measure of proximity between universities. The inverse of geographical distance can be interpreted as a measure of closeness or connectedness between universities, where larger values indicate stronger proximity.
• ENDOGENOUS FEATURES
  • Jaccard coefficient, defined as the proportion of common neighbours in the total number of neighbours. This index is maximum when neighbours are common to both nodes. In co-authorship networks, the Jaccard coefficient catches the idea that nodes having common neighbors are likelier to connect with each other in the next future. In joint project networks this occurrence is however not always true [43].
  • Katz centrality is determined by summing up the contributions of all its neighbors and their neighbors, with each contribution weighted by a factor of decay. Universities with high Katz scores are those that not only have direct collaborations with other universities but also have connections to other universities through their collaborative partners.
  • Closeness centrality, like Betweenness centrality, is a distance-based measure, reflecting the importance of nodes according to their connection distances in the network. It reflects how easily or quickly a node can reach other nodes in terms of geodesic distance (the shortest path between two nodes). Universities with high Closeness centrality scores are considered more central as they have shorter average distances to other universities in the network.
  • Betweenness centrality, referring to the frequency that a university acts as a connection between a pair of other universities. It is computed for all universities every year. A university with higher Betweenness has a larger influence on the whole network and can determine the network’s ability to capture resources and information. Universities with high Betweenness can benefit more from shorter paths towards a larger set of nodes as they are strongly embedded within the network structure.

Optimal prediction via machine learning

We define a learner L_j as a mapping from the set [X, θ_j, λ_j, f_j(⋅)] to an outcome y, where X is the matrix of features, θ_j a vector of estimation parameters, λ_j a vector of tuning parameters, and f_j(⋅) an algorithm taking as inputs X, θ_j, and λ_j. Generally, applied empirical studies use a singleton f_j(⋅) for modeling and predicting targeted outcomes, typically one member of the Generalized Linear Models (GLM) family (linear, probit or multinomial regressions are classical examples). GLM are highly parametric and are not characterized by tuning parameters. Nonparametric models, such as local-kernel, nearest-neighbor, or decision trees are on the contrary characterized by one or more hyper-parameters λ_j which may be optimally chosen to minimize the so-called test prediction error, i.e. the out-of-sample predicting accuracy of the learner.

Fig 2 presents the learning architecture herein proposed. This framework is made of three linked learning processes: (i) the learning over the tuning parameter λ, (ii) the learning over the algorithm f(⋅), and (iii) the learning over new additional information. The departure is in point 1, from where we set off assuming the availability of a dataset [X, y].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The meta-learning machine architecture.

https://doi.org/10.1371/journal.pone.0290018.g002

The first learning process aims at selecting the optimal tuning parameter(s) for a given algorithm f_j(⋅). ML scholars typically do it using K-fold cross-validation, a resampling approach estimating the out-of-sample performance of a learner by leaving one group of observations out of the estimation, and then using prediction over these left-out observations to measure predictive accuracy. This procedure is iteratively repeated for each fold, eventually obtaining K test-accuracy (or, equivalently, test error) measures over which taking the average and the standard deviation.

At the optimal λ_j, one can recover the largest possible prediction accuracy for the learner f_j(⋅). Further prediction improvements can be achieved only by learning from other learners, namely, by exploring other f_j(⋅), with j = 1, …, M (where M is the number of learners at hand).

It is important to observe that the so-called training error, i.e. the in-sample predictive performance of a learner, is a misleading measure of the actual model goodness-of-fit as plagued by the overfitting phenomenon: it may be the case that the training error decreases monotonically with the tuning parameter even if the out-of-sample performance of the learner is worsening. In Fig 2, it corresponds to the light blue sequence of boxes leading to the MSE_TRAIN which is in fact a dead-end node, as not informative for making correct decisions.

Conversely, the yellow sequence leads to the MSE_TEST, which is informative to take correct decisions about the predicting quality of the current learner. At this node, the analyst can compare the current MSE_TEST with a benchmark one (possibly, pre-fixed), and conclude whether to predict using the current learner, or explore alternative learners in the hope of increasing predictive performance. If the level of the current prediction error is too high, the learning architecture would suggest to explore other learners.

In the ML literature, learning over learners is called meta learning, and entails an exploration of the out-of-sample performance of alternative algorithms f_j(⋅) with the goal of identifying one behaving better than the those already explored [44]. For each new f_j(⋅), the learning architecture finds an optimal tuning parameter and a new estimated accuracy (along with its standard deviation). The analyst can either explore the entire bundle of alternatives and finally pick-up the best one, or decide to select the first learner whose accuracy is larger than the benchmark. Either cases are automatically run by this architecture.

The third final learning process concerns the availability of new information, via additional data collection. This induces a reiteration of the initial process whose final outcome can lead to choose a different algorithm and tuning parameter(s), depending on the nature of the incoming information.

As final step, one may combine predictions of single optimal learners into one single super-prediction (ensemble learning). What is the advantage of this procedure? An aggregation of different learners, by reducing prediction variance, can lead to a smaller predictive uncertainty, thus improving overall prediction quality [45].

Feature partial effects for link formation

In this section, we provide evidence about the role played by every single feature viewed as a driver of the probability to create a link. In machine learning, feature importance measures are generally obtained by computing the contribution of each feature to increase the test-accuracy of a learner (or, equivalently, to reduce the test-error). This approach is no doubt informative, but lacks an essential element, i.e. understanding which is the contribution of every single feature to the probability to produce a link. This has to do with the marginal effect (or partial derivative) of a feature on the link probability. As supervised learning is statistically equivalent to estimating the conditional mean of the target y given the vector of features X, defining and computing an average partial (or marginal) effect (APE) is straightforward, with the formula taking on this form: (6) where indicates all the features different from x_j evaluated at their sample mean. Observe that the second equality of Eq (6) derives from the fact that y is binary.

Eq (6) can be easily estimated on the sample, and a simple plot of is sufficient to identify the curvature of its derivative over x_j. In this way, we can derive both the shape of the probability to generate a link as a function of every x_j given that all the other features are held fixed at their mean, and the sign and size of the derivative. In practice, this allows us to gauge whether the link probability increases (decreases) as x_j increases (decreases) and to what amount, and if there are different patterns in different regions of the support of x_j. We consider for this analysis the complete model specification (i.e., both network exogenous and endogenous features).

The APE estimation can be carried out by every single learner, but results may be characterized by a large variability. In other to reduce estimation variance, we opt to rely on an aggregation (average value) of the derivatives obtained learner-by-learner. Aggregating learners’ estimates entails to estimate a “super-learner” generally having smaller variance at reasonable costs in terms of reduced point estimate precision (bias).

The first domain under consideration is the SSH. Fig 9 sets out our estimates of for the SSH sector, that is, the conditional partial expectation of the probability to have a link as a function of every single feature. Observe that, in order to aggregate the features of the different nodes, we use their arithmetic mean.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Conditional partial expectation of the probability to have a link for SSH.

https://doi.org/10.1371/journal.pone.0290018.g009

The total core budget exhibits a decreasing pattern of followed by a stabilization phenomenon, as showed by Fig 10 where the APEs are plotted. We can summarize this finding as follows: initially, if either (or both) universities have a low core funding, it is more likely that they will collaborate in publicly funded research projects. However, as the core funding increases beyond a certain threshold, this probability decreases. This suggests a non-linear relationship between core funding and collaboration probability, where lower levels of core funding foster collaboration, but as the funding increases over a certain threshold, the motivation for cooperation declines.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Pattern of the derivative of the probability of produce a link as a function of the features for SSH.

https://doi.org/10.1371/journal.pone.0290018.g010

The mean citation score presents a U-shaped form: at low level of this index, universities exhibit a high level of APE, that however decreases dramatically near the average of this feature, to then growing towards higher values. Universities having a lower scientific reputation tend to connect together, as well as universities with a higher reputation tend to be more prone to create a link. It is plausible to conclude that organizations tend to choose partners that are similar with respect to their scientific reputation. In this context, the U-shaped pattern indicates a similarity effect.

Universities located in more economically developed regions tend to have a lower probability to lay links. When comparing the findings related to low core funding and low GDP per capita, both demonstrate a similar trend in terms of collaboration probability in publicly funded research projects. In both cases, higher levels of core funding or GDP per capita are associated with a lower likelihood of universities engaging in joint projects. There is evidence of dissimilarity effects in inter-university collaboration, emphasising the importance of bridging regional disparities and fostering collaboration as a means to enhance research and innovation across the EU (these are to be taken however as mere speculations, as we are unable to test such assumptions in this paper). The lower capacity of universities in poorer regions to secure funding, as well as their relatively limited availability of skills and resources, can contribute to the observed dissimilarity effects in inter-university collaboration.

The pattern exhibited by the Betweenness centrality displays a symmetric J-shaped form. At lower level of the Betweenness, the is higher. We then observe, again, a strong drop at relatively high levels, leading to a long plateau at a higher level of Betweenness where it reaches its minimum. This pattern suggests that when universities are poorly central in the network (low Betweenness), i.e. its mediating role is negligible, small increases in its centrality reduces the probability to generate links and, possibly, larger centrality. This effect gets to an end rather soon and fades away along the plateau, until the probability of new links gets locked in. Those nodes with higher Betweenness centrality often connect nodes found in different communities, and usually they are far from each other.

As for university size, we observe a quite clear pattern, as it seems to have a positive correlation with the link formation. The probability of activating a link increases significantly when universities get larger.

The pattern of the Jaccard coefficient is similar to that of the Betweenness centrality; these two features move in the same direction. There may be complementarity effect in this pattern, as well as for the Betweenness centrality. Complementarity is a crucial aspect in collaborations, and is explained by the resource-based view of strategic alliances. Two universities with common neigbours are likelier to share the same scientific cluster, and probably have similar competences. Pairs of universities with a higher Jaccard coefficient have a greater cognitive proximity that can reduce the likelihood of receiving funding, in favor of a diversification of the scientific competences. This result suggests that universities tend to create links with non-neighboring universities (perhaps in different communities). Based on our experiments, we observed that the Jaccard coefficient is a largely effective feature for link prediction. We can observe, compared to the other features, that the range of probability is much higher (from 0 to 0.85). In the context of predicting links between universities in the Horizon 2020 programme, a higher Katz centrality score for a university implies that it has stronger connections and influence within the network. Therefore, universities with higher Katz centrality scores are more likely to have links or collaborations with other universities within their clusters or communities. This analysis can provide valuable information for policymakers, it can aid in identifying universities that may benefit from targeted support and resources to further strengthen their connections and collaborations within the network. The “S” trend observed in Closeness centrality suggests that certain universities within the network exhibit varying degrees of proximity to other nodes. The Closeness centrality scores are relatively low in the start of the trend, indicating that certain universities are quite distant and have a tendency to link less. As the trend progresses, the Closeness centrality scores increase, signifying that the universities become progressively more connected and accessible to others. It highlights the emergence of key universities that act as information brokers, facilitating efficient communication and information diffusion within the network. Finally, the inverse of the distance (proximity) exhibits an oscillating pattern with an initial phase showing a low probability of creating a link between universities that are geographically distant. This is followed by a subsequent increase, then a slight decrease, another rise, and a subsequent descent as the proximity between the universities increases. Overall, understanding the relationship between geographic distance and the probability of link formation provides valuable insights into the dynamics of collaboration in research networks. It highlights the importance of proximity in fostering collaborations, while also acknowledging that other factors beyond distance may influence the likelihood of link formation between universities. Furthermore, it is crucial to consider the nature of the network under analysis, especially in the case of international programmes like Horizon 2020, which emphasize collaborations among institutions from different countries.

For Physical and Engineering Sciences (Fig 11) the pattern shows a lower predicting responsiveness than in SSH, highlighting the limits of using the exogenous variables in the link prediction. Furthermore, we notice here the presence of a greater prediction uncertainty than in the SSH domain. APEs results for PE are showed by Fig 12.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Conditional partial expectation of the probability to have a link for PE.

https://doi.org/10.1371/journal.pone.0290018.g011

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Pattern of the derivative of the probability of produce a link as a function of the features for PE.

https://doi.org/10.1371/journal.pone.0290018.g012

PE, the general pattern of the total core budget displays a plateau in the initial half, indicating that the starting range has the lowest incidence of link creation. The examination of the entire core budget shows an upward tendency in the second section of the distribution. The citation score variable shows a lower probability to form links in second half. Universities in areas with a lower GDP per-capita are likelier to collaborate in publicly funded research projects, after which the tendency begins to decline and ultimately stabilizes. Universities with lower initial Betweenness centrality scores interact less. These couples are more likely to cooperate as the Betweenness centrality grows. In PE network, we show a lower predictive responsiveness as compared to SSH findings. The trend of university size is increasing around the zero point. It is interesting to note, instead, that the Jaccard coefficient shows a very similar trend to that of the SSH sector. Katz centrality reveals a consistent increasing pattern, with a low predictive responsiveness. The “S” pattern in Closeness centrality indicates behavior comparable to the SSH network. The analysis of the inverse of the distance reveals an increasing pattern. It quantifies the degree of proximity or closeness between universities, with higher values indicating closer proximity. Universities that are closer in terms of geographic distance exhibit a higher probability to collaborate.

As for the link prediction in Life Sciences (Fig 13), we observe a decreasing trend in the influence of the total core budget on the probability of collaboration between two universities. This suggests that as the total core budget increases, the likelihood of collaboration between universities decreases. The mean citation score trend is decreasing, with a flattening of the line in the higher values of the variable. GDP per-capita also exhibits a U-shaped curve describing the sub-population of universities contributing to link prediction. There would seem to be a similarity effect between universities regarding the GDP of the regions in which they are located. Universities located in areas with a lower GDP tend to link up more; similarly, universities located in areas with a higher GDP link together. The bridging role of the university on the link creation loses its initial potential with higher values of Betweenness centrality. Our analysis reveals a U-shaped relationship between the size of universities and the probability of collaboration. This U-shaped pattern suggests that there is some degree of similarity in terms of university size that promotes collaboration between universities. However, we observe a low level of responsiveness. This means that the probability of collaboration between universities is not highly sensitive or responsive to changes in university size. The Jaccard index shows similar values to the SSH and PE domains. Moreover, when two universities have low Katz scores, there is an initial high probability of collaboration, followed by an immediate plateau in the probability. We also note a low level of responsiveness of this variable in determining collaboration probabilities. The Closeness centrality exhibits a U-shaped pattern. This finding suggests that nodes with similar values of Closeness centrality in the network tend to exhibit a higher propensity for connecting with each other compared to nodes with intermediate values. Understanding the propensity for nodes with similar Closeness centrality values to connect can provide insights into the underlying dynamics of the network. It suggests that there are certain attractor mechanisms or clustering tendencies that operate based on Closeness centrality values. The inverse of the distance, which captures the geographic proximity between universities, exhibits a trend characterized by low levels of probability for distant universities, followed by an increase in probability up to a certain threshold (around the value of 10), and then a slow decrease.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Conditional partial expectation of the probability to have a link for LS.

https://doi.org/10.1371/journal.pone.0290018.g013

As in the other domains, the Jaccard coefficient has the highest impact on link prediction. Above a certain threshold limit, the probability is steady and closer to zero. The effect of common neighbors in creating connections tends to become saturated in terms of community capacity to offer additional skills. APEs results for LS are showed by Fig 14.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Pattern of the derivative of the probability of produce a link as a function of the features for LS.

https://doi.org/10.1371/journal.pone.0290018.g014

In order to explain the differences emerged between the PE and LS domains compared to the SSH domain in our results, we can refer to the major complexity of the types of organizations and institutional collaborations operating within the projects taking place within the PE and LS domains. These consist of collaborations that often include private companies, of organizations operating through large R&D laboratories that carry out more explorative and risky research, characterized by larger irreversibility and huger sunk costs.

Finally, Figs 15–17 set out the pattern of the elasticity of the probability of forming a link as a function of the features for SSH, PE, and LS respectively. The interpretation of these figures is as follows: for a given feature level , each point in the graph sets out the percentage increase of the probability to set up a link induced by a 100% increase in the feature evaluated at that specific level. These changes can be both positive or negative, and their variability measures the sensitivity of the link probability to changes in the feature itself. By and large, we observe that: (i) negative elasticities are in general likelier than positive ones over all features and domains; (ii) elasticities are more sensitive to changes in the Betweenness centrality, Closeness centrality, and the Jaccard coefficient than in the other features; (iii) the average sensitivity is larger in the SSH, than in the PE and LS domains.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Pattern of the elasticity of the probability of produce a link as a function of the features for SSH.

https://doi.org/10.1371/journal.pone.0290018.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Pattern of the elasticity of the probability of produce a link as a function of the features for PE.

https://doi.org/10.1371/journal.pone.0290018.g016

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 17. Pattern of the elasticity of the probability of produce a link as a function of the features for LS.

https://doi.org/10.1371/journal.pone.0290018.g017