Technology policy formulation has recently gained a renewed importance for governments in the era of strategic competition, but contextual understanding and expertise in deciding where to focus efforts are lacking. As a result, decision-makers might not understand their own national strengths and weaknesses. It’s difficult to judge whether a country’s R&D outputs, no matter how advanced, and its development of production capacity, no matter how significant, align with the country’s intended strategic objectives or can be used effectively to achieve them.

The ability to measure the relative strengths and weaknesses of a country by weighing specific strategic objectives against technical achievements is of paramount importance for countries.

This is especially true as nations seek to resolve supply-chain resilience problems underscored by the Covid-19 pandemic. China’s rejection of the Quad’s vision of a free and open Indo-Pacific, willingness to use economic coercion and the resulting strategic competition, call further attention to multiple technology sectors’ heavy reliance on a single source. A solution must be found that can exploit synergy across multiple technology sectors among collaborating countries while ensuring supply-chain resilience.

Governments’ ability to ensure that strategic objectives pertaining to critical technologies are both well articulated and achievable, and researchers’ and industry’s ability to collaborate in meeting those objectives, would be greatly enabled by the development of an objective and repeatable methodology for measuring technical achievements against clearly defined strategic goals for the critical technology sector. The most pressing challenge should be a relatively straightforward one to resolve: standardise metadata about national objectives and R&D efforts to enable business analysis.

The Quad Critical and Emerging Technology Working Group is an important step towards building collaboration in the research, development and production of critical technologies among like-minded governments. While in nascent stages, the group is gathering momentum and working towards addressing the September 2021 objective to monitor trends in critical and emerging technologies for cooperation, with an initial focus on biotechnology. We recommend as follows:

• Conduct detailed analysis to understand current and emerging gaps in critical and emerging technologies, starting with biotechnology, among like-minded countries.
• Develop a partnership between like-minded countries with advanced technological capabilities to deliver a secure technology supply chain for critical tech. This should include a commitment to a set of core principles for technology development and delivery, including ‘baking in’ democratic principles to the technology and agreeing to share any civilian advances on market terms and refrain from coercion.
• Establish a Quad or Quad Plus critical technologies fund to which participating states pledge investment funds that are then disbursed to address current and emerging critical technologies gaps.

Benchmarking Critical Technologies is a pilot project at ASPI ICPC that examines the development of a handful of critical technologies in the context of strategic partnership and strategic competition.

‘Critical technologies’ broadly refers to strategically important technology areas.¹ Australia, for example, defines ‘critical technology’ as ‘technology that can significantly enhance or pose risks to Australia’s national interests, including our prosperity, social cohesion and national security’.² For this pilot study, we focus on the biotechnology and energy technology sectors in China and in the Quad— the quadrilateral Indo-Pacific diplomatic network consisting of Australia, India, Japan and the US.

This project will be expanded over the course of 2022 to include more technology areas and countries.

During the Quad Leaders’ Summit in March 2021, the Quad Critical and Emerging Technology Working Group was announced. The communiqué from the summit said that the working group was intended to ‘ensure the way in which technology is designed, developed, governed and used is shaped by the Quad countries’ shared values and respect for universal human rights’.³ The communiqué didn’t directly name China, but China was clearly implied in its pledge to recommit to ‘promoting the free, open, rules-based order, rooted in international law and undaunted by coercion, to bolster security and prosperity in the Indo-Pacific and beyond’.⁴

It’s clear that China is the key strategic competitor that the Quad countries are hedging against. They’re technology and manufacturing powerhouses with strong geopolitical influence in the region, which makes the competition both more important and more difficult. As the Quad works to develop capabilities in a range of critical sectors, the Quad members will need to also understand how to leverage each other’s strengths and overcome collective weaknesses to guarantee supply-chain resilience, among other strategic objectives.⁵ They will also need to triangulate the effects of each nation’s digital enmeshment in Chinese supply chains and the net effects of that in particular sectors.

There’s a lack of empirical data to ground decision-makers’ advice on everything from capability gaps to priority investment areas. This project is an attempt to begin to bring additional empirical data to the decision-making process. Our intent is to offer improved clarity on each country’s strengths and weaknesses in each critical technology. After consultation with the Australian Government, we decided to focus on hydrogen energy and solar photovoltaic (solar PV) technologies from the energy sector, and genetic engineering and vaccines and medical countermeasures in the biotechnology sector.

The broader technology areas that these specific technologies sit within are of clear strategic importance. The Quad Leaders’ Summit communiqué established that biotechnology would be the starting point for the Critical and Emerging Technology Working Group’s collaboration. It also highlighted, in the context of the recent COP26 conference, that the Quad would coordinate to ‘establish responsible and resilient clean-energy supply chains’.⁶

To assess national capabilities, we measured each country’s R&D and infrastructure development efforts using patent and patent impact data and academic impact data, and compared those results against the country’s technology-specific policy goals. For patents, we collected two measures for each critical technology: the quantity and quality of the patents. IP Australia provided ASPI ICPC with patent data to analyse the quantity of patents for each critical technology. Additionally, using the commercial product PatentSight developed by LexisNexis, we assessed patent quality with the Patent Asset Index (PAI).⁷ The tool assesses patent quality across various measures in the overall ecosystem of a technology field. Those measures are technology relevance (TR), indicating how much future patents in the field depended upon the patent; market coverage (MC), indicating how much of the global market the patent offers protection of; and competitive impact (CI), the aggregate of TR and MC indicating the economic value of the patent. The aggregate economic value of all patents in the field then constitutes the field’s PAI. For academic impact factors, we used the CiteScore (CS) methodology for measuring impact factors embedded within Elsevier’s Scopus commercial database product.

We also drew on background interviews with industry specialists and senior officials in relevant government departments. Budget data was more challenging to collect, normalise and assess.

Consequently, it isn’t treated as a separate metric, but included with general policy analysis. (For more on our methodology, see the Appendix.)

We recognise that both the policies and technologies on which we base our assessments are evolving. Technology development doesn’t always move in a linear trajectory, and current capabilities aren’t the only indicator of future outcomes. Moreover, the strategic interests and desired policy outcomes one country seeks might not align simply or easily with those of another. Therefore, it isn’t possible to directly compare countries against each other. Rather than arbitrarily rating each country’s progress against the others, we’ve rated each country’s progress in achieving the strategic objectives that it has outlined for each technology area (Figure 1). The progress indicator’s location should be interpreted as being dynamic, given that both policies and technologies will evolve.

Figure 1: Rating scale—country progress in meeting national policy objectives

Rating scale legend

1. Some high-level policy objectives specific to the technology area have been set, but there’s little evidence of efforts making progress towards meeting those objectives.
2. Despite the articulation of some policy objectives pertaining to the technology area, those are still relatively unclear. The country’s R&D and production capabilities don’t appear to be sufficient to contribute to realising the country’s stated policy objectives.
3. There’s some evidence that the country is developing actionable policy in the technology area. There’s clear progress in the country’s ability to contribute to the R&D of the technology, or production capacity. It isn’t clear, however, whether this progress aligns with the country’s stated policy objectives.
4. There’s evidence that stated policy objectives, research and investment are beginning to translate into aligning capabilities.
5. There’s strong evidence that stated policy objectives, research and investment have already translated into aligning capabilities.

Source: Image produced by ASPI.

• Quantity doesn’t mean quality, at least in terms of the way patents and research shift global knowledge and capabilities in the overall ecosystem of a technology field. Our findings on patent impact—measured by how often a patent is cited or purchased—highlighted that China, with the highest number of patent applications filed, didn’t have a correspondingly high impact factor.
  Australia and India, and to a lesser extent Japan, filed far fewer patents, but those few patents had impact more on par with US patents, which were high in both number and impact. One patent can significantly influence the evolution of a technology; others might incrementally advance knowledge or create offshoot fields. Impact factors in these types of analysis can be an objective measure for determining scientific advances or commercial success but aren’t necessarily useful in indicating whether national capabilities support policy objectives. If the point of benchmarking critical technologies capabilities at a national level is to understand what makes a country capable of meeting national policy objectives, competitive in a strategic competition and well placed to work with like-minded partners, then the ability of individual researchers or organisations to advance a technology field doesn’t tell us how competitive a country is in translating concepts to capabilities that align with its strategic objectives. For example, ASPI ICPC believes that in China, the disproportionately large number of patents filed internally is most likely attributable to companies patenting specific applications of technology. In the Quad, countries such as Australia and India have been more impactful for a fewer number of patent applications filed and research papers significantly advance the field.
• Success in connecting policy objectives to outcomes isn’t yet entirely measurable. Our comparison of national policies pertaining to each critical technology we research shows that China, followed by the US, tends to have more clarity about what it seeks to achieve by investing in R&D and production capabilities, and following that up with actions that will achieve those objectives. India, Japan and Australia don’t lack policy development or innovative capacity, but we believe they have been less effective at connecting concepts to capability. This assessment is no doubt at least partially because the development of policy objectives postdates most of our data.
• Metrics don’t explain the context in which innovation is taking place, including incentive structures, and how that affects a country’s ability to meet specific objectives. In China, the incentive structure is designed so that researchers are working to meet specific policy objectives. In fact, companies closely collaborate with the state in technical standards development. According to the revised 2017 Standardisation Law,⁸ the Standardisation Administration of China (an agency under the State Administration for Market Regulation) is required to oversee standards initiation and implementation, and in practice technical committees for standards setting under the Standardisation Administration tend to consist of both companies and research institutes.
  We believe the knock-on effect of the incentive structure in China is that the R&D base is disadvantaged, while companies and researchers focus on implementing specific applications of technology that meet policy needs. China’s National Patent Development Strategy (2011–2020) was designed as a ‘long-term and comprehensive plan to use the patent system and patent resources to enhance the country’s core competitiveness’.⁹ The strategy document prioritises ‘encourag[ing] and supporting[ing] enterprises to upgrade the core technologies and key technologies with patent rights in China’s advantageous fields to national and international standards’.¹⁰ We believe companies are seeking to achieve those objectives by owning the market first, and patents support that approach. They’re adding economic value by increasing the quantity of applications, and owning the market comes before efforts to refine the product. Many PRC-originated technologies are being exported globally (see ASPI ICPC’s Mapping China’s Tech Giants project), no matter what the overall quality of the product in comparison to competitors, and that proliferation is probably achieving some market power and incumbency. It’s a cumulative and individual challenge for the Quad nations to move more rapidly from concept to capability in order to avoid the PRC leading in meeting strategic objectives with that technology.

Readers are warmly encouraged to download the full report to access the detailed sector by sector analysis.

The Patent Search Strategy used in the formulation of the report is available for review here.