The authors have declared that no competing interests exist.

Conceived and designed the experiments: CLB CLM. Performed the experiments: CLB CLM. Analyzed the data: CLB CLM. Contributed reagents/materials/analysis tools: CLB CLM. Wrote the paper: CLB CLM.

The results in this paper establish that information contained in patents in a technological domain is strongly correlated with the rate of technological progress in that domain. The importance of patents in a domain, the recency of patents in a domain and the immediacy of patents in a domain are all strongly correlated with increases in the rate of performance improvement in the domain of interest. A patent metric that combines both importance and immediacy is not only highly correlated (r = 0.76, p = 2.6*10^{-6}) with the performance improvement rate but the correlation is also very robust to domain selection and appears to have good predictive power for more than ten years into the future. Linear regressions with all three causal concepts indicate realistic value in practical use to estimate the important performance improvement rate of a technological domain.

It is possible to quantify the improvement of a technological domain over time, as was first introduced by Moore [_{0} performance at a reference time, t_{0},

The exponential constant (k) is referred to here as the technological improvement rate, which represents the performance improvement over time for a specific generic function that the technological domain is accomplishing. Estimates for k are determined by first constructing a functional performance metric (FPM) that is a measure of the generic function for a technological domain and includes the factors that affect the purchasing decision for artifacts embodying the technology (for example: Watts/$ for Solar PV). Next, data points that measure the FPM are collected over a range of time: a technological improvement rate is determined by an exponential regression vs. time and is statistically analyzed to examine robustness and reliability. While there has been considerable research into finding these improvement rates for different technologies and understanding the best way to measure them [

One of the sources of data that has been widely used for understanding technological change in recent years is patent data [

While many aspects of patents make it an attractive data source for innovation analyses, patents are limited in that they may not cover all inventions or discoveries due to specific patentability criteria that makes it impossible to patent some things (such as Maxwell’s equations) and not all inventions are patented for both economic (secrecy) and competitive reasons (i.e. universities could not collect royalties on patents before 1980). Additionally, the temporal nature of patents can lead to truncation issues with patent data as has been explored by [

Although there is no existing theory that directly attempts to explain the differences between technological improvement rates in technological domains, there are a large number of useful theoretical writings on technological change. This section reviews the technical change literature in order to build upon prior work in Benson and Magee [

HYPOTHESIS 0:

The remainder of this section develops hypotheses based upon various concepts from the literature on technological change. The concepts are operationalized by relationships to specific patent characteristics and the concepts and patent characteristics are summarized in

(1) Simple Patent Count | A: Effort | number of issued US patents in a domain from 1976–2013 |

(2) Average number of forward citations | B: Importance of Patents | average number of times each patent in a domain is cited |

(3) Ratio of important patents | B: Importance of Patents | ratio of patents with cited by over 20 to total patents in a domain |

(4) NPL Ratio | C: Impact of Science | ratio of scientific citations to total citations from the domain patents |

(5) Average publication year | D: Recency | the average date of publication for all patents in a domain |

(6) Average Age of backward citation | E: Immediacy | average age of backward citations for each patent (averaged over the domain) at the time of the citing patents publication |

(7) Price Index (3 years) | E: Immediacy | average proportion of citations that a domain patent receives within 3 years of publication |

(8) Ratio of Backward Citations to Other Domains | F: Breadth of Knowledge | ratio of citations from patents in the domain to patents in other domains |

(9) Mean publication date of backward citations | D & E: Recency and Immediacy | average date of publication for backward citations from patents in a domain |

(10) Average City by within 3 years | B & E: Immediate Importance | average number of citations that a domain patent receives within 3 years of publication |

There are several aspects of technological evolution where the demand or usage could play an important role in the relative rate of improvement in a technological domain. Wright's [

A direct relationship between R&D effort and technical improvement has been discussed by many researchers of technical change. Christensen [

HYPOTHESIS 1:

One of the main explanations of technological change in the literature is based upon categorizing the improvements or inventions within a technology into distinct categories. Many researchers [

The use of forward citations for estimating the importance of a single patent was first suggested on the basis of study of the economic impact of specific patents in a domain (Computed Tomography) relative to other patents in that domain [

Hypothesis 2 seeks to assess the influence of the average importance of patents in a particular domain, with the intuition being that a domain with patents of higher average importance should improve more rapidly than those with lower average importance.

HYPOTHESIS 2:

Hypothesis three involves the impact of particularly important inventions on technological improvement. It is reasonable that technological domains with a larger concentration of very important inventions would improve in performance faster than those with less concentration of such inventions.

HYPOTHESIS 3:

Technology change researchers recognize an essential role for science in technological development; however the complexity of the specific mechanism has continued to unfold. Schumpeter’s early contribution [

To test this idea through patent information, one must connect science directly to patents: some have used a patent characteristic which is the number of backward references to scientific papers [

HYPOTHESIS 4:

The basic intuition underlying concept D is the idea that more rapidly improving domains are newer. Schoenmakers and Duysters [

HYPOTHESIS 5:

The relationship between more immediate science and more rapidly improving scientific fields provides a promising analogy for the importance of immediacy of patents in technological improvement. The connection between immediacy of science and higher scientific improvement rates was suggested by Price [

HYPOTHESIS 6:

There are two ways immediacy can be important. One is the tendency for patents in a domain

HYPOTHESIS 7:

The breadth of knowledge concept reflects combining knowledge from different domains, assuming that the use of information from a larger variety of different sources is likely to result in improved technological outcomes. Rosenberg [

Trajtenberg et al [

HYPOTHESIS 8:

The concepts of recency and immediacy can work together to increase the technological improvement rate. The intuition is that the combination of two independently important drivers will lead to an even stronger effect on the rate of technological improvement through a single combined metric. A metric for recent immediacy that is tested in this paper is the average publication date of all backward citations by patents in a domain. This is directly equivalent to adding the positive linear effects of H5 (patent publication date) and H7 (backward citation age at time of patent publication)

HYPOTHESIS 9:

This hybrid concept combines immediacy and importance and thus argues that domains whose patents are more important in the

HYPOTHESIS 10:

We attempt to explain the variation in k-values (the dependent variable) among domains by the variation in the various patent metrics (independent variables). The objective is to determine which of the patent metrics correlate significantly with the k’s.

There are three main components of the methodology. The first is selecting domains and finding their corresponding k values. For this study, we used the results for 28 domains that are covered in detail by Magee et al [

In the third component of the methodology, the patent sets are analyzed to find the set of patent metrics for each technological domain and are then compared quantitatively with the k-value for each domain. The specifics of calculating the patent metrics for hypothesis testing are now discussed briefly below. The patent set characteristics and the k values for the 28 domains studied are given in Table A (in

Hypothesis 0 is the most general and is tested by the ability of the patent data to explain the differences in technological improvement rates. This hypothesis can be supported by a patent metric that correlates highly with k and has statistical significance. The hypothesis is strongly reinforced by a set of patent metrics that correlate with k that all have statistical significance.

The

Two patent metrics are used to test Concept B and both are directly related to the future (or forward) citations to the patents within a domain. These attempt to measure the impact that a field has on future inventions.

The _{i} is the number of Forward citations for patent

A test of Hypothesis 3 (high frequency of highly cited patents) is the _{i} is the number of Forward citations for patent _{i}

The _{i} is the number of non-patent literature citations for each patent _{i} is the number of backward patent citations for each patent

Hypothesis 5 is evaluated using the

Hypothesis 6 is tested by the _{i} is the number of Forward citations for patent

The immediacy concept is also tested by the _{i} is the number of backward citations for patent

The patent metric that is used to evaluate Hypothesis 8 is the _{i} is the set of backward citations for patent _{i}

Combining the recency and immediacy concepts, it is possible to test a combination of the two using the _{i} is the number of backward citations for patent

The _{i} is the number of Forward citations for patent

The raw patent variables (dates, patent citations, NPL citations) for each of these metrics can be downloaded from

The relationship between a particular patent metric and the k values for all domains was examined graphically as well as statistically.

Contrastingly, ^{6}, indicating that the correlation is quite unlikely to be due to random scattering of the data. The combination of the statistical tests and the visible trend in

All of patent metrics discussed in sections 2 and 3 were tested using this approach and the summary statistics and correlation coefficients are given in

Variable | Mean | SD | Min | Max | (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | (10) |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

(1) Simple Patent Count | 18259 | 29110 | 154 | 149491 | ||||||||||

(2) Average number of forward citations | 11.80 | 3.32 | 6.12 | 22.08 | ||||||||||

(3) Ratio of patents with cited by over 20 | 0.17 | 0.06 | 0.08 | 0.36 | ||||||||||

(4) NPL Ratio | 0.17 | 0.15 | 0.04 | 0.84 | ||||||||||

(5) Average publication year | 2000.7 | 2.9 | 1994.8 | 2006.7 | ||||||||||

(6) Average Age of backward Citation | 10.70 | 3.44 | 6.66 | 18.33 | ||||||||||

(7) Price Index (3 years) | 0.26 | 0.05 | 0.18 | 0.35 | ||||||||||

(8) Ratio of Backward Citations to Other Domains | 0.10 | 0.04 | 0.02 | 0.20 | ||||||||||

(9) Mean publication date of backward citations | 1990.0 | 5.0 | 1981.1 | 1997.8 | ||||||||||

(10) Average forward citations within 3 years | 2.96 | 0.77 | 1.77 | 4.62 | ||||||||||

K-Value correlation with Patent Metric | 0.23 | 0.17 | 0.03 | 0.65 | ||||||||||

Concept A, that effort is an important determinant of relative progress rates among domains surprisingly failed to achieve statistical empirical support. The hypothesis derived from this concept is tested in column 1 above and achieves a p value of. 095: this is above the normal cutoff for statistical significance. On the other hand, Concept B that technological improvement rates are higher in domains with more important/cited patents in a domain is supported. The hypotheses derived from this concept (H2 and H3) are both supported—see columns 2 and 3. The total forward citations (column 2) correlation is 0.48 and has a p value of. 009 which is relatively strong whereas the fraction of patents with more than 20 citations has a more modest correlation of 0.38 with p value of 0.043.

Concept C, which states that domains with closer connections to science improve more rapidly is surprisingly not supported statistically by the results. The test of hypothesis 4 is shown in column 4 of

Concept D—Recency- and hypothesis five that is derived from it (domains with newer patent sets should improve more rapidly) does achieve firm empirical support. The test of this hypothesis is shown in column 5 above and demonstrates strong correlation of 0.54 with a p value of 0.003. Likewise, concept E—Technology improvement is enhanced by increased immediacy of use and knowledge base- is supported strongly. The hypotheses derived from it (H6 and H7) are tested in columns 6 and 7 in

Concept F breadth of knowledge led to H8: domains that cite other domains more frequently will improve more rapidly. This hypothesis is tested in column 8 and does not show any sign of correlation with Cp = 0.11 and p = 0.57. The result of testing the combined recency and immediacy hypothesis is shown in column 9 to achieve a very strong correlation (Cp = 0.72, p = 1.7 x 10^{−5}) with excellent explanatory power. Column 10 tests the hybrid of immediacy and importance and also shows a very strong correlation (Cp = 0.76, p = 2.6x 10^{−6}) with perhaps even more explanatory power. The immediate importance metric has the strongest correlation of any of our patent metrics with the technological improvement rate.

Although seven correlations have p values less than our desired cutoff of 0.05, it is obvious that a number of them contain duplicated information and cannot be useful independently. A very clear example is seen for items 2 and 3 which both are designed as measures of importance and have a cross-correlation near 1 (Cp = 0.96). Not surprisingly, the combined/hybrid metrics have significant cross-correlations with other significant variables. The recent immediacy metric (column 9) shows cross-correlation greater than 0.6 with recency (column 5) as well as both immediacy metrics (6&7) as well as with the immediate importance metric (column 10). The immediate importance (10) metric has correlations greater than 0.6 with both importance metrics (columns 2&3) as well as the backward citation immediacy (column 6), and the recent immediacy metric (column 9), but not the forward citation immediacy metric (column 7). We will return to the issue of overall correlation with multiple regression models shortly but it is useful to first present results concerning robustness of the correlations.

An important issue is whether our 28 domains contain significant selection bias. It is possible that domains we have not yet studied could change our results. Although this concern cannot be fully answered, one way to examine this issue is to look at correlations with smaller subsets of the 28 domains. We proceeded (see supporting material) with a relatively stringent test by randomly separating the set of 28 domains into 2 independent sets of 14 domains (with no domains repeated twice) and the correlation coefficients were re-calculated using only 14 domains each time. This trial was then completed 10 times for a total of 20 different sets of 14 domains and corresponding correlation coefficients. To examine each variable, the mean and standard deviation of the values were calculated, with the signal (r) to noise (sigma) values taken as a measure of robustness.

Patent Metric | Correlation for all 28 domains | Standard Deviation of Correlation for 14 domains | Correlation / Standard Deviation (absolute value) |
---|---|---|---|

(6) Average Age of Citation | -0.59 | 0.103 | 5.678 |

(5) Average publication year | 0.54 | 0.128 | 4.178 |

(2) Average number of forward citations | 0.48 | 0.136 | 3.567 |

(7) Price Index (3 years) | 0.39 | 0.185 | 2.114 |

(3) Ratio of patents with cited by over 20 | 0.38 | 0.200 | 1.923 |

(1) Simple Patent Count | 0.33 | 0.195 | 1.695 |

(4) NPL Ratio | 0.2 | 0.152 | 1.326 |

(8) Ratio of Cites to Own Domains | 0.11 | 0.257 | 0.440 |

Not surprisingly, the correlations with the lowest p values were the most robust to this domain selection test. Given the severity of the test in removing ½ of the domains, there is quite good consistency of the correlations of the metrics on the rate of improvement for each of the metrics with p values < 0.01. In particular, the immediate importance metric of average forward citations within 3 years of publication is remarkably consistent across 20 different correlation tests, indicating that the strength of that signal is not likely to be due to the selection of these specific 28 domains. In the linear regression analysis below, we only use the 5 metrics that are shown to be strongest by this test and by their p values for the entire 28-domain correlation.

The five metrics identified above as showing statistically significant and robust correlation with the k values were included in linear regression models for predicting the technological improvement rate. Numerous regression models were tested using a combination of these variables and the most informative are shown in

Variable/Models | A | B | C | D | E | F | G | H |
---|---|---|---|---|---|---|---|---|

(2) Average number of forward citations | -0.01 | 0.014 | 0.015 | |||||

(5) Average publication year | 0.02 | 0.024 | ||||||

(6) Average Age of Citation | -0.003 | 0.0004 | -0.018 | |||||

(9) Total mean publication date of backward citations | 0.01 | 0.024 | 0.020 | |||||

(10) Average Cited by within 3 years | 0.16 | 0.11 | 0.15 | 0.14 | 0.19 | |||

Intercept | -0.23 | -20.44 | -0.19 | -31.12 | -0.21 | -47.66 | -41.37 | -47.1 |

Model A in ^{2} of 0.53 which indicates that this single variable can “explain” more than ½ of the variation in k across the domains. It is the most powerful of the variables tested and we use it as the basis for Models B through F in ^{2} (0.57) is seen relative to model A, the p values for the coefficient of variable 9 and the intercept indicate that the improvement could well be due to over-fitting. Model C adds the strongest immediacy metric (#6) to the immediate important metric (#10) and similarly improves R^{2} but with p values that make over-fitting a significant concern. Note that the only p values that are strong in both models B and C are for the coefficient for the immediate important metric indicating again the strength of this variable.

Model D combines immediate importance with recency (patent publication date- metric # 5). Despite this variable having the fourth highest correlation with the k-values, it is the first to add significantly to R^{2} (0.64) and does so with p values that make over-fitting unlikely. The combination of the strongest importance metric (#2) with the immediate importance metric is model E and this (like models B and C) gives very modest improvement in R^{2} with p values that raise significant concern about over-fitting. Models F and G leave out the strongest metric (immediate importance) and start with the second strongest (recent immediacy, #9) as the basis. Model F combines the recent immediacy metric and the strongest immediacy metric (average age of backward citation, #6): the p value for the coefficient on metric #6 indicates over-fitting for this variable is very likely. Model G, on the other hand, incorporates the strongest importance variable (forward citations, #2) with the recent immediacy metric (#9) and achieves the (tied for) second best R^{2} along with p values that make over-fitting unlikely. Model H uses neither of the two strongest (hybrid) metrics but instead each of the strongest singular metrics for the three concepts and also achieves the (tied for) second best R^{2} (0.59). Perhaps most interesting is that the p values for

Overall, the results in

An important issue is the ability of the correlations to work in the future not just in the past. A second robustness test examines the predictive capability of the correlations by testing how sensitive the patent metrics correlations were to variations in time. In order to do this, the patent metrics were analyzed for only patents from a variety of time frames that were less than the total time frame. The time frames were analyzed to see how far back from 2013 they could be analyzed and still find similar correlations as the patent metrics show during the entire time frame (1976–2013) and are shown in the supporting information. Ultimately the two strongest and most robust patent metrics are robust to time up to 12 years prior to the experiment reported in detail here, indicating a promising amount of predictive capability.

The major finding of the present study is robust, strong correlations between technological improvement rate and patent metrics for a wide variety of technological domains. An unacceptable interpretation is that the metrics that are strongly correlated with technological improvement rate

As discussed in the literature review supporting hypothesis development, the use of forward citations for estimating importance of a single patent has been well established. The results reported here show that the

Average publication date correlating strongly with technological improvement rate in the variety of domains is also not surprising. Although technology overall being hyper-exponential and thus many rates might increase over time [

The concept of immediacy, first developed by Price [

One of the most important implications of our findings is that patents do contain much information relevant to distinguishing among technological improvement rates in the 28 domains investigated here. Hypothesis 0 is strongly confirmed by the high R^{2} values for the regressions and the multiple strong correlations with patent variables: these findings clearly demonstrate that patents do contain information that is essential to increases in technological improvement rate.

This result is much more aligned with the position that patents are the major data source for technological progress than the contrarian position that patents have very little to do with technological progress. Moreover, analysis of why the explanatory power is not even higher (the R^{2} indicates that more than 1/3 of the variation in k is not explained by combinations of the best variables we have examined) indicates that perhaps only a small part of the issue is lack of information in patents. A Monte Carlo analysis was performed (see ^{2} even with a perfect theory would be reduced to 0.8 to 0.84 due to the imperfect ability to measure k. This indicates that estimating k introduces sufficient noise to account for about ½ of the imperfection found with our model fit to the data. The imperfections in our patent sets representing the domains (62) can diminish the correlations and the possibilities of inconsistent patent writing practices among domains, of better but unknown metrics, for non-linear relationships contributing to imperfect linear correlations and for real effects from textual facts contained in the patents all appear also likely to diminish correlation. Therefore, improvement contributions not captured in patents is definitely less than the contribution of k estimation noise and may not be a significant factor in understanding the imperfections in the regressions.

The results did not support three of the concepts for which we developed hypotheses about their potential influence on the relative rate of performance improvement: effort within a domain, the breadth of knowledge used by a domain and the directness of the science link to a domain are the three unsupported concepts that will each be discussed now. The reasons for the failure to find correlation in each of these cases can be of two kinds: 1) that the concept in fact does not drive

It is a truism that human effort is needed to get any technological progress. However, relatively higher effort

Although breadth of utilized knowledge is a reasonable concept to hypothesize as driving differences in performance improvement among domains, the failure of our test (no sign of correlation) is not as surprising as for the other two failed concepts. This is because a number of tests of breadth of knowledge (on importance of—citations to- individual patents) using various metrics (including number of patent classes per patent) have shown weak and sometimes contrary results [

To question whether science has any impact on technological progress is not a reasonable line of inquiry but the process by which science impacts technology is not yet fully established. Thus, it is not clear that the impact of science should have different impact on performance improvement among domains nor that the impact of science is measured well by citations in patents to scientific articles. Price argued quite early [

Some authors suggest that more heavily cited patents themselves cite more scientific articles [

Overall, it appears that the concept-that breadth of knowledge affects differential improvement rates among domains- is not viable with any metric. On the other hand, we feel that the evidence suggests that the concept- differential science links explain some of the performance rate differential- remains quite viable as a potential explanation despite the failure of our framework to find the effect. The most we can conclude about the third concept- differential effort among domains explain some of the performance rate differential- is that our failure to find such an effect could be due to non-viability of the concept or to metric/framework shortfalls.

One clear implication of the work reported here is that the patent data contains information that can be used to understand the relative rate of improvement among technological domains. The results also strongly support the current practice of using forward citation counts to represent the importance of patents while giving the first indication that importance assessed this way can be extended to entire domains by simple averages across the domains. The work reported here also suggests that little used metrics such as the average patent publication date and the average age of backward citations are quite useful in studying differences among domains. We also introduced two new fairly simple-to- calculate metrics, the average number of forward citations to a group of patents in the first three years after patent publication and the average publication date of the backward citations from a group of patents, that were shown to be particularly powerful in distinguishing among groups of patents. We believe these metrics should be useful to others interested in understanding differences between groups of patents beyond our focus here on understanding the relative rate of progress among a well-defined set of technological domains.

The individual significance of importance, recency and immediacy on the relative rate of progress in technological domains is conceptually significant. Although we did not create any of these concepts, we believe we have distinguished more carefully among them: the empirical work establishes the distinction among these three concepts as meaningful. We suggest that each of these concepts can have causal implications in other technical change phenomena and might fruitfully be more widely studied in other contexts.

The strong explanatory power of models that combine all three concepts also has conceptual implications. A possible connection to prior concepts is with the conceptual frameworks that attribute much of technological change to discontinuities; however, we believe it is important to make the connection with some care. Although not always clearly specified, these concepts often seem to focus on a sharp technological discontinuity whereas our results show that dynamic domains remain such. For our 28 domains, many of the more rapidly improving cases have shown such behavior for more than the 35+ years for which we were able to obtain the corresponding patents and none of these have appreciably yet slowed in performance improvement. A second reason for care is that many of the prior examples of qualitatively selected very important inventions are represented by a large set of patents in this paper- perhaps even a domain such as integrated circuits with its almost 150,000 patents.

The preceding points suggest that a potentially better way to make the connection between technological discontinuities and domains with patents of high importance, recency and immediacy is to assert that the discontinuity of interest is the

One more speculative conceptual contribution is largely based upon the failed correlations as well as the successful ones: we call this concept the rising sea metaphor. Our results show that measurements at the

The technological improvement rate of a domain can be very useful in understanding the potential of a specific technology particularly if one compares it to the improvement rate of competitive and complementary technological domains. This is because the improvement rates are reasonably consistent across time [

The results of the research reported here are correlations robust to the domains analyzed and consistent for 12 years into the future (2001–2013). These findings statistically (in a robust way) reflect what is likely to happen—or at least what is happening now- in performance trends. The process of estimating a technological improvement rate given a domain of interest works as follows:

Select a domain of interest

Use the COM [

Calculate the average number of forward citations in 3 years (column 10 in

Use the predictive model D in

The R^{2} of this predictive model is 0.64, so 64% of the variation in the improvement rate can be explained by the variation in the patent metrics included in the model. This type of estimate can be made in less than 3 hours (at least by an experienced COM user) and is probably nearly as accurate as an estimate that might take more than 30 hours of data search (and might not be possible to find in infinite time). A major implication from the research reported here is the potential to greatly expand the usage of technological improvement rates in technology strategy and research policy. Some useful approaches include:

Quantitatively monitoring improvements at all phases of technological maturity to understand if large (unexpected) changes have occurred.

Monitoring improvement rates in key competing (threat and opportunity) technologies.

The patent based approach to estimation of improvement rates described above can be the basic approach to the monitoring task and it might be applied even very early in the technology’s history possibly even before the start of commercial production as long as sufficient patenting has started.

Often times a competing technology has been used in other application fields and thus improvement rates might be found from actual data but using the patent based approach above would still be useful to improve the robustness of the estimate.

Based upon the prior discussion, relative rates of technical performance increase can have large implications for the future viability of component technologies in products and systems as well as the viability of industries and thus have great importance to forward-looking firms. Acquisition strategy, product component technology choice and appropriate research goals could be informed by improved understanding of the probable improvement potential of relevant technologies. Moreover, the results of performance improvement monitoring have implications for choosing technologies that should receive research funding from firms and governments and for choosing ventures in which to invest risk capital.

This paper represents the first statistically significant comparison between metrics that were derived from individual patent sets from a group of technological domains and the performance improvement rates of the same individual domains. This was done to test hypotheses derived from existing theories of technological change, to initiate predictive theory development and to establish a stronger practical basis for technology strategy and planning for firms and governments. The strong correlations (r = 0.76 for the strongest case) and multiple regressions (R^{2} = 0.64) establish an important empirical finding: patents do contain much significant information relevant to quantitatively determining the differences in technological improvement rates.

The main theoretical implications of the findings reported here are that average importance, recency and immediacy of the patents in a domain each individually drive higher improvement rates and that these concepts are independent enough that models that combine all three are robust predictors of a domains improvement rate. The prediction models apparently provide good evidence of what change is currently happening and meaningful forecasts of the future within the specified robust time frame of 12 years, however past results are not always indicative of future returns and the estimations of the k’s are subject to the same disclaimer. Thus, the potential weaknesses (and possibly unrecognized at present strengths) of the practical application of the results of this research will only be known if and when widespread application occurs.

(DOCX)

We would like to thank Subarna Basnet and Guillaume Baldo, both who worked in the MIT International Design Center, for helping read through the large patent sets to determine the relevancy of the tested patent sets. We would also like to thank Professor Steven Eppinger for giving us useful input on an earlier version of the manuscript and to the SUTD/MIT International Design Center for supporting the research.