Cooperation, competition and the dual-use dilemma in artificial intelligence
What’s the problem?
Despite frequent allusions to a race—or even an ‘arms race’—in artificial intelligence (AI), US leadership and China’s rapid emergence as an AI powerhouse also reflect the reality of cooperation and engagement that extend across the boundaries of strategic competition.1 Even as China and the US, the world’s emergent ‘AI superpowers’,2 are increasingly competing in AI at the national level, their business, technology and research sectors are also deeply ‘entangled’ through a range of linkages and collaborations. That dynamic stems from and reflects the nature of AI research and commercialisation—despite active competition, it is open and often quite collaborative.3 These engagements can, of course, be mutually beneficial, but they can also be exploited through licit and illicit means to further China’s indigenous innovation and provide an asymmetric advantage.4 The core dilemma is that the Chinese party-state has demonstrated the capacity and intention to co-opt private tech companies and academic research to advance national and defence objectives in ways that are far from transparent.
This has resulted in a ‘dual-use dilemma’ in which the openness that’s characteristic of science and innovation in democracies can result in unforeseen consequences, undermining the values, interests and competitiveness of the US, Australia and other like-minded nations in these strategic technologies.5 These ‘entanglements’ have included ties between US tech firms and Chinese partners with military connections,6 as well as cooperation between Australian universities and the Chinese People’s Liberation Army (PLA).7 Despite the genuine advantages they may offer, such problematic partnerships can also result in the transfer of dual-use research and technologies that advance Chinese military modernisation, perhaps disrupting the future balance of power in the Indo-Pacific, or facilitate the party-state’s construction of surveillance capabilities that are starting to diffuse globally.
These adverse externalities have troubling implications for US military advantage, authoritarian regime resilience and even the future of democracy.8 How should policymakers balance the risks and benefits of such entanglement,9 while enhancing competitiveness in this strategic technology?
What’s the solution?
These unique and complex dynamics require a range of policy responses that balance the risks and benefits of these partnerships, collaborations and engagements. To enhance situational awareness, policymakers should examine closely research, academic and commercial partnerships that may prove problematic, and then consider updates and revisions to national export controls, defence trade controls and investment review mechanisms as targeted countermeasures. While there is a rationale for visa screening of foreign nationals who plan to study or research sensitive technologies, restrictions should be imposed only on the basis of evidence of direct and clear connections to foreign militaries, governments or intelligence services,10 and scrutiny should focus more on organisations engaging in talent recruitment that are linked to the Chinese central and local governments or to the Chinese Communist Party (CCP). At the same time, there are compelling reasons to sustain scientific cooperation, with safeguards for risk mitigation, including transparency and the protection of sensitive data.
Critically, the US and Australia must pursue policies that actively enhance the dynamism of their own innovation ecosystems to ensure future competitiveness. It is vital to bolster declining support for science and commit to increasing funding for basic research and the long-term development of strategic technologies. Given the criticality of human capital, governments should prioritise improving the accessibility and affordability of STEM education at all levels, while attracting and welcoming talent through favourable immigration policies. In this quest for competitive advantage, the US and Australia must also pursue closer public–private partnerships and expand alliance cooperation on defence innovation.
AI ‘without borders’
Today, national competition in AI is intensifying at a time when the engine for technological innovation in such dual-use technologies has shifted from governments to commercial enterprises. In today’s complex, globalised world, flows of talent, capital and technologies are rapid, dynamic and not readily constrained by borders. Chinese investments and acquisitions in Silicon Valley—and US investments in China—are sizable and increasing, despite intense concerns about the security risks of such investments,11 which have motivated reforms to the Committee on Foreign Investment in the United States (CFIUS) and could result in discretionary implementation of China’s national security review mechanism in response.12 This increased globalisation of innovation ecosystems has proven beneficial to AI development, and dynamic US and Chinese companies are emerging as world leaders in the field.
Increasingly, these enterprises are quite international in their outlook, presence and workforce while engaging in a global quest for talent.13 For the time being, the US remains the centre of gravity for the top talent in AI, and Silicon Valley is the epicentre of this talent ‘arms race’.14 While currently confronting major bottlenecks in human capital, China has great potential, given the number of graduates in science and engineering and the range of new training and educational programs dedicated to cultivating AI talent.15 At the same time, the Chinese government is actively incentivising the return and recruitment of ‘strategic scientists’ via state talent plans.16 At the forefront of the AI revolution, Baidu and Google epitomise in their strategic decisions and activities the linkages and interconnectivity among such global centres of innovation as Silicon Valley and Beijing.17
Baidu has prioritised AI and has emerged as a leading player in this domain. It created the Institute for Deep Learning in Beijing in 2013 and then established its Silicon Valley Artificial Intelligence Laboratory (SVAIL), which employs about 200 people, in 2014.18 Baidu’s CEO, Li Yanhong (李彦宏, or Robin Li), advocated as early as 2015, prior to the Chinese Government’s decision to prioritise AI, for a ‘China Brain’ plan that would involve a massive national initiative in AI, including welcoming military funding and involvement.19
Increasingly, Baidu has actively invested in and acquired US AI start-ups, including xPerception and Kitt.ai,20 while seeking to expand its US-based workforce. The company has stated that Silicon Valley ‘is becoming increasingly important in Baidu’s global strategy as a base for attracting world-class talent.’21 In March 2017, Baidu announced plans to establish a second laboratory in Silicon Valley, which is expected to add another 150 employees.22 Notably, Baidu has also launched the Apollo project, which is a collaborative initiative to advance the development of self-driving cars that involves more than 100 tech companies and automakers, including Ford, NVIDIA, and Microsoft.23 At the same time, Baidu is engaged in research on military applications of AI, particularly command and control.24
Google remains at the forefront of AI development, leveraging an international presence and global workforce. Beyond Silicon Valley, Google has opened AI research centres in Paris, New York and Tokyo,25 and it will soon add Beijing and then Accra, Ghana.26 When Google announced the opening of the Google AI China Center in December 2017, chief scientist Fei-Fei Li declared, ‘I believe AI and its benefits have no borders. Whether a breakthrough occurs in Silicon Valley, Beijing, or anywhere else, it has the potential to make everyone’s life better for the entire world.’27 She emphasised, ‘we want to work with the best AI talent, wherever that talent is, to achieve’ Google’s mission.28
Google’s decision to expand its presence and activities in China, after withdrawing its search product from the market due to concerns over censorship, surveillance and the theft of intellectual property via cyber espionage in 2010,29 reflects this enthusiasm for the potential of future talent in China—and probably the availability of a sizable market and massive amounts of data as well.30 At the same time, this decision presents an interesting counterpoint to Google’s recent issuing of a statement of principles that included a commitment not to build technologies used for surveillance.31 Given the dual-use nature of these technologies, Google’s choice to engage in China may involve risks and raise ethical concerns,32 especially considering the Chinese party-state’s agenda for and approach to AI.
China’s global AI strategy and ambitions
At the highest levels, the Chinese Government is prioritising and directing strong state support to AI development, leveraging and harnessing the dynamism of tech companies that are at the forefront of China’s AI revolution. The New Generation Artificial Intelligence Development Plan (新一代人工 智能发展规划), released in July 2017, recognised this strategic technology as a ‘new focal point of international competition’, declaring China’s intention to emerge as the world’s ‘premier AI innovation centre’ by 2030.33 The Three-Year Action Plan to Promote the Development of New-Generation Artificial Intelligence Industry (促进新一代人工智能产业发展三年行动计划) (2018–2020), released in December 2017, called for China to achieve ‘major breakthroughs in a series of landmark AI products’ and ‘establish international competitive advantage’ by 2020.34 China’s central and local governments are providing high and ever-rising levels of funding for research and development on next-generation AI technologies, while seeking to create a robust foundation for innovation by introducing new talent and education initiatives, developing standards and regulatory frameworks, and supporting the availability of data, testing and cloud platforms.35
China’s ambition to ‘lead the world’ in AI is self-evident.
China’s ambition to ‘lead the world’ in AI is self-evident.36 These plans and policies should be contextualised by its tradition of techno-nationalism and current aspirations to emerge as a ‘science and technology superpower’ (科技强国).37 In recent history, indigenous Chinese innovations, particularly defence technological developments, have been advanced and accelerated through licit and illicit means of tech transfer, including extensive industrial espionage.38 However, pursuing a new strategy of innovation-driven development,39 China is actively seeking to progress beyond more absorptive approaches to innovation and instead become a pioneer in emerging technologies, including through increasing investment in basic research.40 To further this agenda, the Chinese government is avidly targeting overseas students and scientists, offering considerable incentives via talent plans and engaging in recruitment via ‘talent bases’ and organisations that are often linked to the CCP or to central or local governments.4142
At this point, the success of these initiatives remains to be seen, and there are even reasons to question whether an AI bubble may arise due to excessive enthusiasm and investments. Although China’s future potential for innovation shouldn’t be dismissed or discounted, this ‘rise’ in AI often generates alarm and exuberance that can distract from recognition of major obstacles that remain. As its plans openly admit, China continues to lag behind the US in cutting-edge research and is attempting to compensate for current shortfalls in human capital.43 Notably, China confronts continued difficulties in the development of indigenous semiconductors,44 which will be critical to the hardware dimension of future advances in AI,45 despite billions in investment and quite flagrant attempts to steal intellectual property from US companies.46
While gradually becoming more capable of truly independent innovation, China also intends to coordinate and optimise its use of both domestic and international ‘innovation resources’.47 Notably, the New Generation AI Development Plan calls for an approach of ‘going out’ (走出去) involving overseas mergers and acquisitions, equity investments and venture capital, along with the establishment of R&D centres abroad.48 For instance, a subsidiary of the China Electronics Technology Group Corporation (CETC), a state-owned defence conglomerate, established an ‘innovation centre’ in Silicon Valley in 2014, which seeks to take advantage of that ecosystem with a focus on big data and other advanced information technologies.49 In Australia,50 CETC established a joint research centre with the University of Technology Sydney (UTS), which will focus on AI, autonomous systems and quantum computing, in April 2017.51 Starting in 2018, CETC’s Information Science Academy is also funding a project at UTS on ‘A Complex Data Condition Based Public Security Online Video Retrieval System’, which could have clear applications in surveillance.52 There have been extensive collaborations on dual-use AI technologies between PLA researchers from the National University of Defence Technology and academics at UTS, the University of New South Wales and the Australian National University.53
Meanwhile, Huawei is actively funding research and pursuing academic partnerships in the US and Australia, including through its Huawei Innovation Research Program.54 China’s ‘One Belt, One Road’ strategy is also concentrating on scientific and technological cooperation, including educational exchanges and research partnerships, such as a new Sino-German joint AI laboratory.55 Some of these new collaborations will focus on robotics and AI technologies, often enabling access to new sources of data that may facilitate China’s emergence as a global leader in AI development.56 In certain instances, China’s provision of funding to these initiatives may also reorient the direction of research based on its own priorities.57
As China seeks to advance indigenous innovation, the strategy of ‘going out’ is complemented by a focus on ‘bringing in’ (引进来) to ensure that vital talent and technologies are drawn back into China.58 At the same time, the Chinese government is evidently seeking to ensure that innovation ‘made in China’ will stay in China. As the US undertakes reforms to CFIUS, China could respond by recalibrating the implementation of its own national security review process, which is ambiguous enough to allow for great discretion in its application, pursuant to an expansive concept of national or state security (国家安全).59 Notably, the State Council has also issued a new notice that requires that scientific data generated within China be submitted to state data centres for review and approval before publication.60 The policy purports to promote open access to and sharing of scientific data within China, while creating ambiguous new restrictions that, depending upon their implementation, could render future cooperation asymmetrical in its benefits.61 Given these factors, while opportunities for research cooperation should often be welcomed, it is also important to ensure transparency regarding the research and intellectual property that may result from it, as well as the security of valuable or
China’s integrated approach to indigenous innovation
In pursuit of its dreams of AI dominance, China is pioneering a new paradigm of indigenous innovation that takes advantage of critical synergies through creating mechanisms for deeper integration among the party-state, technology companies and the military. The CCP seeks not only to support private Chinese companies in their quest for innovation but also to control and guide them, ensuring that the companies serve the needs of the party and don’t become a threat to it. China’s ‘champions’ in AI— Baidu, Alibaba, Tencent and iFlytek—are at the forefront of innovation in the field, and this ‘national team’ will be supported and leveraged to advance state objectives and national competitiveness.62
For instance, Baidu is leading China’s National Engineering Laboratory for Deep Learning Technologies and Applications (深度学习技术及应用国家工程实验室),63 and iFlytek is leading the State Key Laboratory of Cognitive Intelligence (认知智能国家重点实验室).64 It seems likely that the research in these new laboratories will be directed to dual-use purposes. These champions will also undertake the development of new open innovation platforms in AI: Baidu will be responsible for autonomous vehicles, Alibaba Cloud (Aliyun) for smart cities, Tencent for medical imaging and iFlytek for smart voice (e.g., speech recognition, natural-language processing, machine translation, etc.).65 The platforms will be piloted in the Xiong’an New Area, a development southwest of Beijing that’s intended to be a futuristic demonstration of Chinese innovation and to showcase AI technologies and applications in action.66
Meanwhile, Xi Jinping has recently reaffirmed the Mao-era sentiment that ‘the party leads everything’, and China’s advances in AI must also be understood in the context of this system, in which the CCP is steadily increasing its control over private companies.67 In recent years, the CCP has introduced representatives of party branches and committees into notionally private companies,68 which have started to undertake more active ‘party building’ (党建) activities that are intended to expand the CCP’s presence and influence.69 Just about every major tech company, including Baidu, Alibaba, Tencent, Sohu, Sina and NetEase, has a party secretary, who is often a fairly senior figure within the company, and new requirements may even require all listed companies to ‘beef up party building’.70 For example, in March 2017, the CCP Capital Internet Association Commission (中共首都互联网协会 委员会) convened a party committee expansion meeting and a work meeting on grassroots party building that brought together the leaders of many prominent companies.71 At the meeting, Baidu Party Secretary Zhu Guang (朱光), who is also a Senior Vice President responsible for public relations and government affairs,72 talked about innovation in ‘party building work’, including the development of a mobile solution for ‘party building’. He committed Baidu to leveraging its capabilities in big data and AI applications, as well as its ‘ecological advantage’, to enhance the effectiveness of such efforts.73
This blurring of the boundaries between the party-state and its champions may create a tension between national strategic objectives and these companies’ global commercial interests.74 Increasingly, the CCP is even attempting to extend its reach into, and authority over, foreign companies operating in China.75
The dual-use dilemma in China’s AI development
The future trajectory of AI in China will inherently be shaped and constrained by the interests and imperatives of the party-state, and international collaboration with Chinese research institutions and corporate actors needs to be understood, and engaged in, with this important context in mind. Critically, AI will enhance both economic development and military modernization, while reinforcing the party’s ability to control its population through domestic surveillance, all of which are integral to the regime’s security and legitimacy. China’s AI plans and policies include the concern that AI will remain ‘secure and controllable’ (安全 , 可控), given the risks of societal disruption, while highlighting the importance of AI ‘to elevate significantly the capability and level of social governance, playing an irreplaceable role in effectively maintaining social stability’, thus bolstering regime security.76
Indeed, the pursuit of such ‘innovations’ in social governance through big data and AI has included the construction of predictive policing and surveillance capabilities, often developed with the assistance of start-ups such as SenseTime and Yitu Tech, that have often been abused, particularly in Xinjiang.77 Given the party’s attempts to extend its reach—and the trend towards deeper integration in civilian and military AI efforts in China—it can be difficult to disentangle notionally commercial activities from those directly linked to the party-state’s agendas for social control, indigenous innovation and military modernisation.
... a national strategy of ‘military–civil fusion’...
China seeks to take full advantage of the dual-use nature of AI technologies through a national strategy of ‘military–civil fusion’ (军民融合). This high-level agenda is directed by the CCP’s Military–Civil Fusion Development Commission (中央军民融合发展委员会) under the leadership of President Xi Jinping himself.78 Through a range of policy initiatives, China intends to ensure that advances in AI can be readily turned to dual-use applications to enhance national defence innovation. Although the effective implementation of military–civil fusion in AI may involve major challenges, this approach is presently advancing the creation of mechanisms and institutions that can integrate and coordinate R&D among scientific research institutes, universities, commercial enterprises, the defence industry and military units.79 For instance, in June 2017, Tsinghua University announced its plans to establish a Military–Civil Fusion National Defence Peak Technologies Laboratory (清华大学军民融合国防尖端技术实验室) that will create a platform for the pursuit of dual-use applications of emerging technologies, especially AI.80 Notably, in March 2018, China’s first ‘national defence science and technology innovation rapid response small group’ (国防科技创新快速响应小组) was launched by the CMC Science and Technology Commission in Shenzhen,81 and is intended to ‘use advanced commercial technologies to serve the military.’82
China’s AI ‘national champions’ may often be engaged in support of this agenda of military-civil fusion. Notably, in January 2018, Baidu and the 28th Research Institute of the China Electronics Technology Group’s (CETC), a state-owned defence conglomerate, established the Joint Laboratory for Intelligent Command and Control Technologies (智能指挥控制技术联合实验室), located in Nanjing.83 The CETC 28th Research Institute is known as a leading enterprise in the development of military information systems, specializing in the development of command automation systems,84 and it seeks to advance the use of new-generation information technology in defence ‘informatization’ (信息化).85
This partnership is directly linked to China’s national strategy of military-civil fusion, leveraging the respective advantages of CETC and Baidu to take advantage of the potential of big data, artificial intelligence, and cloud computing. Going forward, the new joint laboratory will focus on increasing the level of ‘intelligentization’ (智能化) in command information systems, as well as designing and developing new-generation command information systems ‘with intelligentization as the core.’ Baidu’s involvement in this new laboratory reflects its active contribution to military-civil fusion, a strategy that is resulting in a further blurring of boundaries between commercial and defence developments.
Policy considerations and recommendations
There is no single or simple solution, and policy responses must take into account the inherent complexities of these global dynamics, which necessitate highly targeted and nuanced measures to mitigate risk.86 At the same time, real and serious concerns about China’s exploitation of the openness of our democracies must not lead to reactive or indiscriminate approaches that could cause collateral damage to the inclusivity and engagement that are critical to innovation.
The benefits of scientific collaboration are compelling, and continued cooperation should be supported, with appropriate awareness and safeguards. In future, the quest to achieve an advantage in emerging technologies will only intensify, and the US and Australia must also look to enhance their own competitiveness in these strategic technologies.87
The options for policy response include, but aren’t limited to, the measures detailed below.
Strengthen targeted, coordinated countermeasures.
1: Review recent and existing research and commercial partnerships on strategic technologies that involve support and funding from foreign militaries, governments or state-owned/supported enterprises, evaluating the dual-use risks and potential externality outcomes in each case.
- Evaluate early-stage research to determine the likelihood that it may turn out to have disruptive dual-use implications in the future.
- Present a public report with findings and recommendations to raise awareness and ensure transparency.
- Continue to push back against forced tech transfer in joint ventures.88
2: Explore updates and revisions to national export controls, defence trade controls and investment review mechanisms that take into account the unique challenges of dual-use commercial technologies; communicate those updates clearly and publicly to relevant stakeholders.
- Share lessons learned and pursue coordination with allies and partners to account for the global scope and scale of these dynamics.
- Ensure that these restrictions are applied to sensitive datasets associated with AI development, including data used for training purposes.
3: Engage in visa screening of foreign nationals who plan to study or research sensitive or strategic technologies, targeting scrutiny on the basis of whether or not students or researchers have direct and clear connections to foreign militaries, governments or intelligence services.
- Deny visas to those who are determined to be likely to leverage their studies or research in support of a foreign military that is not a security partner.
- Incorporate an independent review mechanism into the process to assess evidentiary standards and mitigate risks of bias in visa determinations.
4: Identify organisations engaging in talent recruitment that are linked to the Chinese central and local governments or to the CCP, and require their registration as foreign agents where appropriate.
5: Enhance counterintelligence capabilities, particularly by augmenting language and technical expertise.
Encourage best practices and safeguards for risk mitigation in partnerships and collaborations, with a particular focus on universities.
6: Introduce stricter accountability and reporting requirements, managed by departments of education, which make transparent international sources of funding for research strategic technologies
7: Engage in outreach to companies, universities and think tanks in order to highlight the potential for risk or unintended externalities in joint ventures and partnerships, including through developing and presenting a series of case studies based on past incidents.
8: Propose best practices for future academic collaborations and commercial partnerships, including transparency about the terms for scientific data and intellectual property, as well as clear standards on ethics and academic freedom.
- Identify favourable domains to sustain open collaboration and engagement, such as issues of safety and standards.
9: Introduce, or where appropriate adjust, policies or guidelines restricting those who work for national or military research institutes and laboratories or receive public funding at a certain level from organisations accepting funding from or collaborating with a foreign military, state-owned enterprise or ‘national champion’ that is not an ally.
Go on the offensive through policies to enhance national competitiveness in technological innovation.
10: Increase and commit to sustaining funding for basic research and the long-term development of AI technologies.
11: Prioritise improving the accessibility and affordability of STEM education at all levels, including creating new scholarships to support those studying computer science, AI and other priority disciplines.
12: Sustain openness to immigration, welcoming graduating students and talented researchers, while potentially offering a fast-track option to citizenship.
13: Pursue closer public–private partnerships through creating new incubators and institutions that create a more diverse and dynamic community for innovation.89
- Encourage dialogue and engagement between the tech and defence communities on issues of law, ethics and safety.
14: Explore the expansion of alliance coordination and cooperation in defence innovation, including collaboration in research, development and experimentation with new technologies and their applications.
15: Engage with like-minded nations to advance discussions of AI ethics and standards, as well as potential normative and governance frameworks.