The Defense Advanced Research Projects Agency (DARPA) was created in 1958 in response to the Soviet Union’s launch of Sputnik. The agency matured in an era of incredible American scientific and technological advancement characterized by coordination between academia, industry, and government. Today, facing economic and technological challenges from China, the United States has reacted by launching “ARPAs for everything”: the recently established ARPA-H (for health), a newly authorized ARPA-I (for infrastructure), and various Focused Research Organizations (FROs), such as Speculative Technologies, Schmidt Futures’ Convergent Research, and Actuate, which develop small and midsize technology projects.
Yet the goal-oriented research at the heart of an “ARPA approach,” whether at a government ARPA or FRO, is not a panacea for all that ails US innovation. As former DARPA program managers, our time at the agency exposed us to the best of what the research community can achieve with focused goals and adequate support. We witnessed the power of funding exploratory, high-risk ideas that pushed the boundaries of the possible. But since leaving DARPA, our interactions with the commercial sector and broader research community have convinced us that the United States has lost sight of its true strategic advantage: its ability to adopt and spread innovations, or “diffusion capacity.”
At DARPA we viewed innovation competition through the lens of evolving, complex adaptive ecosystems, examining the critical role of knowledge diffusion, feedback loops, and adaptations to accelerate learning, exploit opportunities, and spread innovations for competitive advantage. From this vantage point, a country’s diffusion capacity determines its competitiveness. Political scientist Jeffrey Ding has shown that diffusion capacity, even more than the ability to achieve breakthroughs, is essential for sustaining economic growth and productivity gains.
Ding’s research highlights the late 1800s, when the United States struggled to create innovations as quickly as Europe’s great powers but excelled at diffusion, which led to remarkable economic growth. Today, China has a diffusion deficit stemming from its top-down economic framework and information controls, similar in some ways to the former Soviet Union’s. Thus the United States and its partners will need to make the most of this structural advantage by finding ways to support and increase the nation’s capacity for diffusion.
A FRACTURED INNOVATION ECOSYSTEM
Diffusion occurs when scientific discoveries are translated into application and adoption, both within and across sectors, and importantly, when the need to improve existing products sparks further scientific exploration. In this sense, diffusion can be understood as a two-way exchange between innovation and consumers, fostering swift progress across multiple domains. A high-diffusion environment is evident when discoveries can be rapidly exploited for a diversity of target applications, or conversely, when a company is presented with a diversity of exploitable technology options to select from when pursuing competitive advantage. When a breakthrough innovation hits a high diffusion environment, there is a tangible quickening in the rhythm of progress—as in the early periods of the smartphone app ecosystem, or with the global spread of CRISPR gene-editing technology.
Cycles of diffusion and innovation tend to reinforce themselves: advances by one company encourage both imitation and diversification by competitors. High rates of new business creation using entirely new technologies as well as uptake of new products and methods are signs of an economy with strong diffusion capacity.
Here it’s important to distinguish between technological innovation that can drive long-term productivity improvements and economic growth, and business model innovation. Recently, US venture capital has focused much attention on the latter, exemplified by the software-as-a-service transformation behind Salesforce or the two-sided taxi marketplace of Uber. However, business model innovation often explicitly limits diffusion by seeking to create customer lock-in and high switching costs—think Gillette’s invention of a business model featuring a cheap razor and a more profitable lifetime bill of blades. This approach can create great businesses and loyal customers or even reshape whole industries, but it has different impacts on productivity than technological innovation.
As these examples suggest, the US innovation ecosystem comprises a vibrant set of actors, but its incentive structures are fractured in a way that subverts diffusion. For diffusion to occur, innovations must spread beyond isolated researchers, startups, and the occasional risk-taking corporation to an entire network of participants and, ultimately, more conservative consumers. Not only are business incentives often crosswise to diffusion, but there are also substantial disconnects between sectors that prevent sources of innovation, capital, and know-how across the ecosystem from truly engaging each other.
The first of these disconnects occurs within the highly balkanized environment of academic research. In our experience with DARPA’s Defense Sciences Office, some of the most exciting outcomes happened when we put together groups of academic researchers that would typically never interact, such as computer scientists and anthropologists, with military operators. But the reality is that most academic research findings are known only to other academics in increasingly esoteric subfields. Traditional research funding mechanisms incentivize individuals over teams and high citation counts per invested dollar—not translation to impact. The limited diffusion between the academic research enterprise and industry that does occur is facilitated by university technology transfer and licensing offices, but income from these efforts is only indirectly used to fund follow-on research—and so gains do not build upon each other.
Conventional wisdom suggests that venture capitalists (VCs) will help to bridge this disconnect, but the gap in understanding between academics and VCs remains significant. Moreover, the venture model—which seeks extremely high returns on invested capital in relatively short periods of 5 to 10 years—is not equally suited to all technologies. Investors have thrived in software because founders can pick a product, rapidly iterate for product-market fit, and look for signs of exponential growth. Venture investment has also found utility in biotech, which has a clearly defined market and technical risk assessed through government approval. In the past 20 years, the portion of top-selling drugs developed outside the walls of big pharma has risen from 15% to 80%. In these sectors, VC investment has indeed contributed to a range of both technological and business model innovations.
The gains in hardware and physical science sectors, however, have been more limited. For example, the venture model failed during the clean energy boom of the late 2000s, when a mix of unclear tech-market fit, long development times, higher risk, and much higher capital intensity led to poor returns and a collapse in VC investment by 2016. Some alternatives to the conventional VC model did develop as a result, characterized by exceptionally patient capital aimed at bridging gaps in the US innovation ecosystem. These alternatives, including Breakthrough Energy Ventures, operate more like philanthropies and are backed by the likes of Bill Gates and Eric Schmidt. Unfortunately, this model is tough to replicate—there are only so many multibillionaires—and most of the capital that powers VC and private equity lacks the patience and risk tolerance these alternatives afford. So new technology enterprises that seek to scale up still face a “capital stack” problem: they are too expensive for venture backing and too risky for more conventional loans or corporate debt. Acquisition then becomes an attractive “exit” goal for investors, but this may actually stifle diffusion by absorbing new knowledge into a single corporation’s innovation system.
This brings us to one of the most significant paradigm shifts in the US innovation ecosystem affecting diffusion: the demise of the corporate lab. Fifty years ago, corporate labs contributed significantly to diffusion, because they served as critical nodes for assembling nascent technologies, business needs, and scientific and engineering talent. But with a few notable exceptions, including Alphabet’s X (which bills itself as “the moonshot factory”), corporate labs are largely defunct today. Though corporate research and development spending is higher than ever, most of that investment is focused on incremental innovation to existing products, which shareholders, executives, and potential inventors see as safer bets.
Today’s dominant corporate focus is R&D productivity: maximizing the likelihood of new revenue with minimal spending, ideally in timeframes under three years, by leveraging existing supplier and partner networks. Some “open innovation” organizations have cropped up to fill the vacuum left by the old corporate lab model, but even these are confined to scouting technically mature solutions to incremental innovation problems. Thus, at the critical juncture between research and industry, the incentives are more aligned with locking up innovation than diffusing it.
AMERICA’S STRUCTURAL ADVANTAGE
Although the United States has most of the ingredients of an ecosystem ripe for fast-paced innovation diffusion, we contend that present incentives embedded in the system are just as likely to thwart diffusion as to catalyze it. To effectively leverage the country’s inherent advantages, leaders need to recalibrate these incentives, transforming them into accelerants of innovation diffusion rather than dampeners. We recommend focusing on three leverage points: what to work on, who is involved, and how to make it happen.
The first thing to consider when incentivizing innovation diffusion is what technology to focus on. The current trend of government subsidies for technologies characterized by incumbent lock-in and massive facilities that require significant capital expenses—such as semiconductor fabrication—is inefficient. Instead, policymakers should focus on catalyzing the development of modular technologies that foster further innovative discovery, subvert dependence on particular technology trajectories and supply chains, and address critical industrial and economic needs.
Going modular could transform the whole US innovation ecosystem in important ways. In a recent paper in Science on innovation for decarbonization, Charlie Wilson, Arnulf Grubler, Nuno Bento, and coauthors argue that technologies that are “small in size, low in cost, many in number, and distributed in application” may be the key to breaking the inertia of fossil energy systems. In energy, this is visible with the rapid progress in solar power and battery storage. In the authors’ words, “granularity is associated with faster diffusion, lower investment risk, faster learning, shorter lifetimes, lower complexity, larger efficiency potentials, more equitable access, more job creation, and higher returns on innovation investment.”
We have seen the benefits of modular technology repeatedly in our post-DARPA work: accessible components enable the rapid growth of a broad range of new technologies including artificial intelligence, synthetic biology, and low-Earth-orbit spacecraft. It’s also present in DARPA’s history, as when Very Large-Scale Integration—embedding huge amounts of transistors onto a single semiconductor chip—enabled a shift from handcrafted chip design to a more distributed and modular design process, opening the door to an influx of new innovators and a diversity of semiconductor applications. Modular technologies reduce investment risk, increase supply chain resilience, and facilitate adaptability, so enterprising entrepreneurs can build solutions to fit their local context.
The second opportunity to catalyze rapid diffusion requires looking at who does it—the human capital. Today’s innovation system incentivizes deep specialization. As a result, “lateral thinkers” who can pull technical ideas between one sector and another or from research to application are rare. In our time in DARPA leadership, recruiting for these profiles was one of our toughest challenges. While a few institutions value a diffusion-oriented mindset, most universities penalize attention not directed to publication, and most companies can’t calculate a return on investment for curiosity.
Cultivating talent for diffusion requires supporting more opportunities for cross-sectoral experience across the research enterprise. Targeting a portion of National Institutes of Health and National Science Foundation funding toward rotational programs could be transformative: deploying academic researchers into industry settings for a year, and reciprocally offering industry researchers paid sabbaticals for academic research, could facilitate cross-pollination that enhances diversity of perspectives and fosters a culture that prizes curiosity and cross-disciplinary thinking in both sectors. A century ago, the Massachusetts Institute of Technology’s Technology Plan integrated industrial consulting into the research agenda for faculty, which provided unique exposure to industrial work for students and created a funding stream for academic departments. This Practice School program, of which one of us (Paschkewitz) is an alumnus, remains a unique model that should be more broadly replicated. We are aware of how difficult it would be to get employers onboard, but as long as there is a stigma about developing cross-industry careers that span academic research and application, the shortage of diffusion-oriented talent will remain a problem not only for ARPAs, but for the entire US innovation system.
Finally, our recommendations on how to catalyze diffusion are based on updating some old ideas so the United States can build on its existing structural advantages in entrepreneurship and finance. Most importantly, we suggest targeted government policy interventions on key sticking points that prevent diffusion today.
For a strong model of a government organization facilitating effective diffusion, look to the way the National Advisory Committee for Aeronautics (NACA) contributed to the growth of the US aerospace industry in the 1920s and ’30s. As a federal agency, NACA generated foundational research on topics such as airfoil performance and inlet design, which it disseminated in forms that were directly usable by industry engineers: blueprints, data tables, parametric representations, calculations, and experimental validation. These outputs inspired the academic research community and fostered a rich, supportive environment for the aerospace industry to leverage cutting-edge research for practical applications. Though NACA did not directly finance industry players or seek to directly impact market outcomes, its research efforts offered significant indirect financial benefits. As a government entity with a clear mandate for research and knowledge dissemination, it provided stable infrastructure (including facilities like wind tunnels and testbed aircraft) and institutional support for aerospace technology innovation. By undertaking fundamental research and making the results widely available, it effectively subsidized the research costs for the aerospace industry, thus encouraging innovation and experimentation.
We recommend a similar approach, updated for the twenty-first century, to accelerate technological advancements in materials, computing architectures, the bioeconomy, and clean energy. Such catalyst institutions would have far stronger diffusion capacity and efficiency than the more common research consortia, which are motivated by pooling risk and minimizing spillovers. Following the NACA model, catalyst institutions should commit to practical application in their charter and success metrics; propose modular building blocks; and use modern tools and architectures for sharing research products to amplify diffusion within and between government agencies, national labs, and other institutions traditionally engaged in standards and research.
A further challenge is accelerating the diffusion of competitive technologies across the value chain of suppliers, producers, distributors, and users. This can be done by developing collaboration organizations that both support cooperative R&D and craft incentives to sustain US strategic advantage. A reference model here is the Semiconductor Manufacturing Technology consortium, or SEMATECH. Formed in response to the competitive pressures of the global semiconductor market in the late 1980s, SEMATECH brought together major US semiconductor companies and the US Department of Defense. Through cooperative R&D efforts, SEMATECH members could solve common technological challenges and thereby nurture the diffusion of innovation within the ecosystem.
Today, a SEMATECH-like collaboration organization could be used to align US strategic interests in the development of 5G/6G telecommunications technology. A critical aspect of the competition between the United States and China is the degree to which solutions are vertically integrated or modularized. The Chinese telecommunications company Huawei’s vertical integration of the radio access network (RAN) hardware and network infrastructure that underpin 5G networks offers early advantages for efficient deployment and cost savings. In contrast, emerging US players are developing an alternative, modular approach called Open RAN that decouples the hardware and software components of 5G to enable long-term innovation, flexibility, and competition in design. This approach faces initial challenges in integration and optimization, but a collaboration organization could enable coordination between the government and the private sector. It could define a new 5G/6G strategy that helps US players shape the value chain and plays to American strengths on virtualized, software-centric systems—while providing opportunities to work with international partners and allies. It’s worth noting that these collaboration hubs could also provide fertile ground for the incubation and spread of modular technologies by promoting licensing, sharing reference architectures, and providing test infrastructure.
A final challenge is aligning financial incentives to accelerate diffusion. Today’s constellation of public funding, VCs, private equity, corporations, and investment banks allows many promising innovations to fall through the gaps. And too great a proportion of government money is concentrated in early-stage investment. The ecosystem needs more incentives and financing for technology scale-up and value chain transformation. The potential impact of such a strategy can be seen in the Inflation Reduction Act, which aims to create markets by establishing stronger incentives for risk-taking action through tax credits or investment subsidies. This is already paying off: carbon capture and US-based battery manufacturing have moved from certain losses to potential profit opportunities.
Additional financial incentives can be leveraged to encourage diffusion. To create a demand-side incentive for new technology-driven products, the government could use advance market commitments to guarantee the purchase of a certain quantity of a new product before it exists—as it did for COVID-19 vaccines. And, to encourage venture capital to invest in commercializing innovation with longer-term payoffs, tax law could be changed so that the holding period for long-term capital gains rates is extended from three years to five years, with incentives for even longer holding periods contingent on the sale of equity remaining in the United States or its allies.
Blended finance is another tool that can combine private capital seeking a return with government coinvestment or loan guarantees. This preserves an investor’s potential upside while limiting downside risk, aiming to draw new investment into preferentially selected sectors. The World Bank has examined how blended finance has been used to support innovative off-grid solar projects in frontier markets, which would traditionally face obstacles from investors unfamiliar with the technology or market. In this case, providing long-term debt to private investors on favorable terms helps underwrite and mitigate risks for these pioneering energy projects.
Despite the concern over the United States’ relative innovation advantage and handwringing over the need for new ARPAs, a handful of policy interventions and incentive adjustments can catalyze diffusion for a new wave of transformative technologies. Importantly, these improvements would capitalize on profound and long-held US strengths, leveraging a culture that promotes the free interchange of ideas and valorizes entrepreneurial drive, and is supported by the world’s most dynamic private capital system. Consider the role that the free exchange of ideas has played in the development of the open-source movement, for example, which underpins so much of the modern technology economy.
It is these advantages in diffusion capacity that the United States should amplify. Building a long-term competitive edge in diffusion will be more effective than an imitative model that seeks to create national champions. China’s recent move to force closures of international consultancies, including Forrester Research, Mintz Group, Bain & Company, and Capvision, which has limited foreign parties’ access to Chinese industry information, signals that the country’s leaders are willing to compromise diffusion capacity—and their ability to tap into a diverse global knowledge base.
In contrast, the United States can embrace its chaotic strengths in service of diffusing more rapid innovation. This will yield far greater gains than merely multiplying the ARPA model. Rather than building walls around US fundamental research, policymakers should invest in the nation’s superior ability to exploit innovation by catalyzing collaboration. With a diffusion-centric approach, the United States can ensure its leadership while driving innovation that reaches far and wide.