The Innovation Red Queen's Race
Discover more from Risk+Progress
The Innovation Red Queen's Race
Innovation is getting harder, why?
“Everything that can be invented has been invented,” claimed the Commissioner the US patent office, Charles H. Duell. Actually, this famously erroneous 1899 statement, may very well have been misquoted, but over a century later, it runs the risk of slowly becoming true. Innovation is getting a lot harder and the pace of change is perhaps poised to slow down. This fact has profound implications for our future.
Getting Harder
While the reasons are debatable, it is undoubtedly true that innovation is becoming increasingly difficult. Indeed, there is mounting evidence documenting a relentless and systemic decline in research productivity. In many areas of science, it now takes far more capital, both in terms of financial and human capital, to advance forward.
The evidence is wide-ranging: The cost of developing new pharmaceuticals now doubles every nine years, it takes 18 times as many engineers to fulfill one cycle of Moore’s Law than it did in 1971, and agriculture research productivity for seed yields is declining by about 5 percent per year. Studies analyzing millions of published scientific papers and patents have also come to the conclusion that they are becoming less and less disruptive over time.
Indeed, Bloom et. al. came to a stark conclusion: Research productivity across the board is declining at a rate of around 5.3 percent per year on average. Since invention is a driving force behind economic growth itself, this means that the economy has to double research efforts every 13 years just to maintain the same rate of overall economic growth.
The most common explanation for this phenomenon is that the “low-hanging fruit” of discovery has already been picked. Thus, only the “harder” discoveries remain to be found. These more difficult discoveries engender more knowledge, push the limits of science harder, and require increasingly greater capital to uncover. Science, it is said, could be something of a Red Queen’s race, where we must run ever faster just to keep the current pace.
Misplaced Incentives?
Jay Bhattacharya and Mikko Packalen, argue that the low-hanging fruit hypothesis may not tell the whole story. Instead, they suggest that truly groundbreaking research is not rewarded in the same way that it used to be. Instead, they posit that a heavy emphasis on citations is changing discovery incentives in a way that encourages incremental innovation over breakthrough discoveries.
Since the 1970s, there has been a growing expectation that researchers not only publish frequently but also publish works that are frequently cited by others (have a large impact). Together, these metrics, known as the “h-score,” are a kind of “batting average” for researchers. The more a researcher publishes and the more his work is cited, the more likely he is to obtain grants and further his career.
Bhattacharya and Packalen claim that overreliance on citations as a measure of success creates a perverse incentive. They present a simple model of the lifecycle of a scientific idea. In this model, the x-axis represents scientific effort, while the y-axis measures its relative impact.
Most ideas and discoveries have little impact, at least not until they cross a “breakthrough” threshold (see above), if they ever do. By conducting one’s research in areas in crowded fields where that breakthrough has already occurred, that is, doing incremental work rather than risking exploratory science, researchers are more likely to have their work cited by others.
From Hot Springs to Covid
The problem is that many breakthrough discoveries trace their roots to basic exploration that are often not recognized by the scientific community until later on, sometimes decades later. In 1967, at a hot spring near Yellowstone National Park, biologists discovered bacteria that could live and reproduce at extremely hot temperatures. The findings were published but were little noticed.
When the bacteria were isolated in the 1970s, it led to an understanding of an enzyme called taq polymerase, which can amplify DNA at high temperatures. In the 1980s, biochemists utilized this knowledge and came up with the idea of applying a cell’s ability to repeatedly copy DNA to a specific strand of DNA. This technology, which we now call polymerase chain reaction (“PCR”), won the Nobel Prize and ultimately revolutionized biomedicine.
PCR technology enabled faster and cheaper DNA analysis and make the Human Genome Project possible. It also became crucial to fighting the Covid-19 pandemic as “PCR” tests became the gold standard for detecting the virus. All of this would have been impossible if it were not for relatively obscure research made decades earlier in hot springs.
Bhattacharya and Packalen suggest that to re-enliven the sciences, we have to alter the incentive structures placed upon researchers. Instead of boiling their work down to the number of citations or an h-score, we need a more multi-dimensional approach that also rates the novelty of their work. By giving greater weight to novel and exploratory work, researchers will be more willing to take risks in unproven areas of science.
This isn’t easy. Bhattacharya and Packalen recommend that this would require indexing word sequences that appear in published scientific papers. The “vintage” of these ideas can then be measured based on how long ago they first arose in the literature. Having determined the “vintage” of the idea, we can score papers across two dimensions, scientific impact as measured in citations and its “edge” score, or how novel that work is.
Why it Matters
Innovation underpins economic growth and human prosperity. A slowing pace of innovation, should it continue, means a less prosperous future for everyone as it makes it less likely that complex technological issues, such as climate change, and other global challenges are overcome.
The rising cost of innovation is becoming more salient against a backdrop of global depopulation, which simultaneously threatens to stifle our ability to supply human and financial capital to research. In the context of climate change, we risk stunting progress while civilization is hopelessly dependent on fossil fuels.
As I discuss here, we may be able to mitigate this trend by designing policies that encourage more scientists and more capital to flow toward research. An abundance agenda for innovation would be a prudent, and perhaps essential, response to the upcoming slowing pace of innovation.
The Innovation Red Queen's Race
Apart from the "low hanging fruit" factor and the citation issue, it seems likely that innovation is being slowed by the accretion of regulations and bureaucracy in all institutions. This is clearly the case in government agencies (often obstacles to innovation) and in universities, less so in companies -- but companies often have to work through regulations and bureaucracies to develop and use an innovation.