Tag Archive for: censorship

The TikTok boomerang

Few predicted that TikTok users in the United States would flock to the Chinese app RedNote (Xiaohongshu) in defiance of a US government ban. And yet in the space of just two days this week, RedNote became the most downloaded app in the US, gaining 700,000 users—most of them American TikTok refugees.

Since US data security was the rationale for the TikTok ban, American users’ migration to other Chinese apps only amplifies those concerns. Unlike TikTok—a platform that does not operate in China and is not subject to Chinese law—RedNote is a domestic Chinese app bound by strict Chinese regulations. Moreover, while TikTok says that it stores US user data exclusively within the US, with oversight by a US-led security team, RedNote stores its data entirely in China.

In recent years, China has introduced a series of data protection laws ostensibly aimed at safeguarding user information. But these regulations primarily target businesses, imposing far fewer constraints on government access to personal data. Chinese public authorities thus have wide discretion in requesting and accessing user data.

Beyond the issue of data privacy, US authorities also worry that TikTok might be used to influence public opinion in the US. But TikTok’s algorithms are closely monitored by Oracle, as part of a deal to address security concerns. In contrast, RedNote’s algorithms operate under the close scrutiny of the Chinese government, and the app is subject to China’s stringent content-moderation requirements, which could further shape the opinions of the TikTok refugees now flocking to the platform.

Given the rationale for the law banning TikTok, it is hard to imagine RedNote escaping similar scrutiny. Now that the US Supreme Court has upheld the TikTok law, the president will have the authority to designate RedNote as a national security threat, too. But this process may quickly descend into a game of Whac-a-Mole. As US users migrate from one Chinese platform to another, regulators will find themselves locked in an endless cycle of banning Chinese apps.

As the list of banned apps grows, the US risks constructing its own Great Firewall—a mirror to the censorship strategy long employed by China. Even if Chinese apps are removed from US app stores, tech-savvy users can easily bypass such restrictions with VPNs, just as Chinese users do to access foreign platforms. That means the US government will soon confront the limits of its ability to ban Chinese apps.

Moreover, each new restriction risks fueling defiance, driving even more users toward Chinese-controlled platforms. Instead of mitigating national security concerns, this strategy may inadvertently exacerbate them, introducing the kinds of vulnerabilities that the original ban was supposed to address.

The TikTok ban thus puts the US government in a near-untenable position, which may explain why Donald Trump is reportedly weighing options to spare TikTok (despite having initiated the ban during his first term).

Yet reversing the ban carries its own risks. As legislation passed by congress, it cannot be repealed by executive order. In theory, Trump could direct law enforcement agencies not to enforce the ban; but that would have far-reaching consequences, not least by calling into question America’s commitment to the rule of law (again mirroring a charge the US has long leveled against China).

An alternative to banning TikTok is a forced divestiture of the app’s US operations, but that solution hinges on one critical factor: China’s approval. In 2020, China implemented restrictions on the export of technologies such as recommendation algorithms—the core of TikTok’s operations—effectively giving the Chinese government veto power over any potential deal.

The TikTok dilemma thus now serves as a powerful bargaining chip for China’s leaders, granting them significant leverage in their dealings with Trump, who campaigned on a promise to impose higher import tariffs on Chinese goods. Not surprisingly, he turned to Chinese President Xi Jinping for help just hours before the Supreme Court was set to weigh in on the ban.

At the same time, the TikTok saga has handed China yet another strategic gift. Friendly interaction between TikTok refugees and Chinese netizens on RedNote has created an unprecedented opportunity for cultural exchange, something China’s rulers have long aspired to but struggled to achieve.

For more than two decades, the Chinese government has aggressively tried to promote its culture and expand its influence in the US. But while it has purchased ads in Times Square and established Confucius Institutes on US university campuses, these efforts have largely failed to gain traction. Remarkably, what RedNote has achieved in just a few days seems to have eclipsed the cumulative impact of all these prior initiatives.

As I explored in my recent book, High Wire, centralised decision-making frequently results in fragile, rather than resilient, regulatory outcomes. The TikTok saga offers a stark reminder that an over-concentration of presidential power in shaping US foreign policy—particularly toward China—can lead to similar outcomes. With Trump expected to consolidate executive power, surround himself with loyalists and operate with fewer institutional constraints during his second term, this trend seems likely to intensify, generating vast unintended consequences.

Editors’ picks for 2024: ‘China may be putting the Great Firewall into orbit’

Originally published on 26 August 2024.

The first satellites for China’s ambitious G60 mega-constellation are in orbit in preparation for offering global satellite internet services—and we should worry about how this will help Beijing export its model of digital authoritarianism around the world.

The G60’s inaugural launch on 5 August 2024 carried 18 satellites into low-Earth orbit (LEO) on a Long March 6A rocket. Led by Shanghai Spacecom Satellite Technology and backed by the Shanghai Municipal Government, the project aims to compete in the commercial satellite internet market with SpaceX’s Starlink, providing regional coverage by 2025 and global coverage by 2027.

The G60 is one of three mega-constellations that China is planning, alongside the Guowang project, run by state-owned China Satellite Services, and the Honghu-3 constellation, led by Shanghai Lanjian Hongqing Technology Company. These constellations provide the infrastructure to support China’s rapidly growing commercial space sector, including its satellite internet initiatives which are making rapid advances.

China launched the world’s first 6G test satellite into LEO in January. GalaxySpace recently made headlines by deploying satellite internet services in Thailand, the first time Chinese LEO satellite internet had been deployed overseas. In June, the Chinese company OneLinQ launched China’s first civilian domestic satellite internet service, indicating it would expand through countries that had signed up to China’s Belt and Road Initiative.

Yet through these efforts, China is not only securing its position in the satellite internet market but laying the groundwork for expanding its digital governance model far beyond its borders.

Central to China’s ambition is the concept of cyber sovereignty—the notion that each nation has the right to govern its digital domain. In practice, China has used this principle to build a heavily censored surveillance system supporting the Chinese Communist Party’s power, widely condemned for violating human rights.

China’s satellite internet services would enable other governments to adopt similar practices, as the nature of satellite internet makes it susceptible to state control.

Satellite internet is more controllable due to its centralised infrastructure, where data is routed through a limited number of ground stations or gateways. This enables censorship and surveillance as service providers and authorities can more easily monitor, block and filter content.

In contrast, traditional internet infrastructure relies on a decentralised network of sub-sea cables and terrestrial networks managed by many stakeholders with thousands of data exchange points. This decentralised structure makes it difficult for any entity to exert complete control over the flow of information, as countries such as Russia — which initially welcomed the open internet, unlike China or North Korea—have learned.

Countries that use China’s satellite internet service providers could more easily control what information is accessible within their borders, much as the Great Firewall of China operates domestically. This could mean blocking politically sensitive topics, monitoring user activity, or shutting down the internet during unrest. While satellite internet has often been hailed as a means for dissidents and activists to bypass restrictive governments, the reality under China’s model, which would place it in the hands of nation-states, would be starkly different.

China is already exporting its digital authoritarianism through such initiatives as the Digital Silk Road, providing technologies and governance models that enable censorship, surveillance and social control to other countries. These efforts come amid a rise in the global spread of authoritarianism as governments seek to exert control over online spaces. Adopting Chinese satellite internet services would accelerate this trend, empowering other countries to implement similar controls and restrict human rights globally.

Offering satellite internet worldwide has other benefits for Beijing. Countries relying on China’s infrastructure for connectivity may risk being pressured to comply with Beijing’s demands, including censoring content critical of China, sharing sensitive data or suppressing domestic dissent in China’s interests. For example, a journalist in a country that relies on China’s satellite internet services might find his or her connection reduced or severed when reporting human rights abuses in China.

The centralised nature of satellite internet may also make countries more vulnerable to cyber espionage by the Chinese government or malicious actors. Chinese satellite providers may also be subject to China’s stringent data localisation policies, such as the Cybersecurity Law, which requires companies to store data within China and make it accessible to the Chinese government. As China’s satellite projects are intended to provide global coverage, the data of international users—spanning communication, location, and internet activity—would be subject to Chinese data laws. Chinese authorities could potentially access any data transmitted through Chinese satellite internet services.

The global deployment of China’s internet satellite services is still some way off and faces significant challenges. However, if China’s satellite internet services are adopted, the world may witness the rise of a new digital Iron Curtain extending from space, dividing the free flow of information and imposing state control on a global scale.

China may be putting the Great Firewall into orbit

The first satellites for China’s ambitious G60 mega-constellation are in orbit in preparation for offering global satellite internet services—and we should worry about how this will help Beijing export its model of digital authoritarianism around the world.

The G60’s inaugural launch on 5 August 2024 carried 18 satellites into low-Earth orbit (LEO) on a Long March 6A rocket. Led by Shanghai Spacecom Satellite Technology and backed by the Shanghai Municipal Government, the project aims to compete in the commercial satellite internet market with SpaceX’s Starlink, providing regional coverage by 2025 and global coverage by 2027.

The G60 is one of three mega-constellations that China is planning, alongside the Guowang project, run by state-owned China Satellite Services, and the Honghu-3 constellation, led by Shanghai Lanjian Hongqing Technology Company. These constellations provide the infrastructure to support China’s rapidly growing commercial space sector, including its satellite internet initiatives which are making rapid advances.

China launched the world’s first 6G test satellite into LEO in January. GalaxySpace recently made headlines by deploying satellite internet services in Thailand, the first time Chinese LEO satellite internet had been deployed overseas. In June, the Chinese company OneLinQ launched China’s first civilian domestic satellite internet service, indicating it would expand through countries that had signed up to China’s Belt and Road Initiative.

Yet through these efforts, China is not only securing its position in the satellite internet market but laying the groundwork for expanding its digital governance model far beyond its borders.

Central to China’s ambition is the concept of cyber sovereignty—the notion that each nation has the right to govern its digital domain. In practice, China has used this principle to build a heavily censored surveillance system supporting the Chinese Communist Party’s power, widely condemned for violating human rights.

China’s satellite internet services would enable other governments to adopt similar practices, as the nature of satellite internet makes it susceptible to state control.

Satellite internet is more controllable due to its centralised infrastructure, where data is routed through a limited number of ground stations or gateways. This enables censorship and surveillance as service providers and authorities can more easily monitor, block and filter content.

In contrast, traditional internet infrastructure relies on a decentralised network of sub-sea cables and terrestrial networks managed by many stakeholders with thousands of data exchange points. This decentralised structure makes it difficult for any entity to exert complete control over the flow of information, as countries such as Russia — which initially welcomed the open internet, unlike China or North Korea—have learned.

Countries that use China’s satellite internet service providers could more easily control what information is accessible within their borders, much as the Great Firewall of China operates domestically. This could mean blocking politically sensitive topics, monitoring user activity, or shutting down the internet during unrest. While satellite internet has often been hailed as a means for dissidents and activists to bypass restrictive governments, the reality under China’s model, which would place it in the hands of nation-states, would be starkly different.

China is already exporting its digital authoritarianism through such initiatives as the Digital Silk Road, providing technologies and governance models that enable censorship, surveillance and social control to other countries. These efforts come amid a rise in the global spread of authoritarianism as governments seek to exert control over online spaces. Adopting Chinese satellite internet services would accelerate this trend, empowering other countries to implement similar controls and restrict human rights globally.

Offering satellite internet worldwide has other benefits for Beijing. Countries relying on China’s infrastructure for connectivity may risk being pressured to comply with Beijing’s demands, including censoring content critical of China, sharing sensitive data or suppressing domestic dissent in China’s interests. For example, a journalist in a country that relies on China’s satellite internet services might find his or her connection reduced or severed when reporting human rights abuses in China.

The centralised nature of satellite internet may also make countries more vulnerable to cyber espionage by the Chinese government or malicious actors. Chinese satellite providers may also be subject to China’s stringent data localisation policies, such as the Cybersecurity Law, which requires companies to store data within China and make it accessible to the Chinese government. As China’s satellite projects are intended to provide global coverage, the data of international users—spanning communication, location, and internet activity—would be subject to Chinese data laws. Chinese authorities could potentially access any data transmitted through Chinese satellite internet services.

The global deployment of China’s internet satellite services is still some way off and faces significant challenges. However, if China’s satellite internet services are adopted, the world may witness the rise of a new digital Iron Curtain extending from space, dividing the free flow of information and imposing state control on a global scale.

Putin’s quest to disconnect Russia from the global internet

In late June, in the aftermath of the failed Wagner Group rebellion, the Kremlin moved swiftly to censor social media and scrub the Russian internet of details of the attempted coup. The digital crackdown has continued in the months since, as President Vladimir Putin has sought to reconsolidate control. But at the same time, another, less-publicised endeavour also took place—the Kremlin embarked on a test to disconnect Russia from the global internet.

The exercise, which took place on 2 July, tested Russia’s ‘sovereign internet’ and its ability to act independently. According to a statement released by Russia’s Ministry of Communications, the test was a success. Roskomnadzor, the government agency responsible for controlling Russian mass media, worked with telecommunications companies and internet service providers to disconnect the country from the global internet for a few hours.

While the statement claimed that the test wasn’t in preparation for a permanent disconnection, Russia has been attempting to create a sovereign internet—or ‘RuNet’—for years.

Since the birth of the internet, the Russian government has had an uneasy relationship with the technology, balancing the economic benefits of the open internet and access to the global economy with concerns about the security of the regime. A turning point came in 2011, when mass protests prompted the Russian government to increase its control over online platforms.

Moscow also stepped up its international advocacy for the concept of cyber and internet ‘sovereignty’, pushing for more state control over the governance of the internet in forums such as the United Nations. These efforts—supported by China and other authoritarian countries like Iran and North Korea—have so far had limited success, prompting Putin to explore how Russia could independently break away from the global internet architecture.

The passage of Russia’s controversial Sovereign Internet Law in 2019 provided the government with the legislative authority to create a ‘sustainable, secure and autonomous’ online environment and establish the technical means to create a ‘sovereign Russian internet’. The law empowered the government to establish a national domain name system (DNS) to route and monitor internet traffic within its borders and mandated annual tests to ‘disconnect’ Russia from the global internet architecture—just like the one conducted in July.

Unsurprisingly, the new legislation caused an outcry—both in Russia and internationally—about creeping censorship and surveillance and the stifling of free speech in Russia. The only saving grace at the time was that, despite the law, the Russian government didn’t really have the technical means or the political will to completely disconnect from the global internet.

Russia’s previous tests to disconnect from the internet have had little success. Russia is heavily reliant on the global economy and Western technology platforms and its population expects engagement with the outside world. So even if Russia could technically disconnect from the global internet, it couldn’t afford to get cut off from the rest of the world.

However, the situation changed after Russia began its full-scale invasion of Ukraine in February 2022.

The West acted quickly to put sanctions in place, cutting Russia off from the global financial system. In response, Putin banned access to Western social media and technology companies like Facebook, Twitter (now X) and Instagram. Independent news websites were blocked and the Kremlin increased its censorship and enhanced its ability to monitor internet traffic and block access to websites or topics that it deemed ‘illegal’.

In March 2022, Ukraine asked the Internet Corporation for Assigned Names and Numbers (ICANN), the non-profit group that oversees the global DNS, to effectively disconnect Russia from the internet by revoking Russian domains and shutting down DNS root servers in Russia. ICANN rejected the request. It doesn’t have the technical ability to take down domains, and wouldn’t even if it could since its role is to uphold the free and open internet.

ICANN’s response goes to the heart of the challenges Russia faces in trying to create a truly sovereign internet.

Built as a decentralised network of networks, the global internet is designed to allow information to flow freely across borders. The multi-stakeholder model of the internet—with civil society, industry, governments and international bodies all playing a role—ensures that no single entity, including governments, can wield absolute control over the entire digital realm.

This, of course, is based on the assumption that a country is connected to the global internet. North Korea has never been physically connected, and China created a domestic network with a careful selection of limited entry and exit points to the global internet from the very beginning.

In contrast, Russia initially welcomed the internet with open arms, leading to a complex web of interconnections that now complicates its pursuit of internet sovereignty. This makes it almost impossible to pinpoint and regulate every entry and exit point through which digital data flows.

Russia therefore has to take a different approach to China and North Korea. Central to Russia’s efforts is the manipulation of the processes and protocols dictating internet traffic—the DNS, often referred to as the internet’s address book. Russia’s approach involves creating a localised DNS system that directs citizen internet traffic solely within Russia’s geographic boundaries. By redirecting traffic to approved sites hosted inside Russia, the Kremlin aims to reduce the need to filter and monitor external information from the broader internet. This would allow Russia to effectively attain a sovereign internet while not completely disconnecting from the global internet architecture.

While there’s no reason to be confident in Russia’s ability to undertake such a genuinely complicated and complex endeavour, the successful test in July, and the recent news that Russia is preparing to block VPNs (virtual private networks), suggest that Putin is inching closer to his goal of effectively creating a sovereign internet. If he is successful, it will not only usher in an even more repressive digital environment in Russia but will serve as an attractive model for other countries to follow.

Holding up a mirror in Australia–China relations

In November 2021, almost one year after he had posted a computer-generated image on Twitter of a grinning Australian soldier appearing to slit the throat of an Afghan child, Chinese foreign ministry spokesperson Zhao Lijian bragged about the success of his tweet at the inaugural China Internet Civilization Conference in Beijing.

‘With this tweet, we put Australia firmly in the dock and let the world know about the heinous crimes committed by Australian troops in Afghanistan,’ Zhao told his audience.

The point of the seminar at which Zhao spoke these words was, remarkably, how China can create a ‘credible, lovable and respectable image of China’. That warm-and-cuddly sounding goal had been laid out by President Xi Jinping in May 2021 in a politburo study session about how China can improve its external propaganda.

But any suggestion that it signalled that Xi was putting his wolf-warrior diplomats on a leash was soon put to rest. Zhao told his audience that his tweet was an example of Beijing ‘taking the initiative to set issues’ and fighting an ‘active war’ for international public opinion.

‘The Prime Minister of Australia held a press conference after we posted the tweet and asked us to apologise and to delete the tweet. He also asked Twitter to delete it,’ Zhao boasted.

‘Instead of deleting the tweet, I pinned it. Twitter didn’t delete the image and text, but they hid it behind a warning, which only made people more curious to know what the hidden image was.’

There’s scant evidence that the aggressive approach of China’s wolf-warrior diplomats changes any hearts and minds outside mainland China. Quite the opposite, in fact. Yale scholars Daniel Mattingly and James Sundquist exposed more than 3,000 English-speaking web users in India to real messages from Chinese diplomats and found that the more aggressive posts didn’t have broad appeal.

But from Zhao’s perspective, the tweet was a resounding success. In addition to causing Australia’s prime minister to publicly and, as it turned out, impotently protest the tweet, Zhao told his audience in Beijing that it went on to prompt three days of questioning from the media at the daily Ministry of Foreign Affairs press conferences.

The point of Zhao’s tweet was to distract, and it succeeded. As Zhao told his audience, it’s better for Beijing to set the terms of the debate rather than fall into any rhetorical traps laid by its perceived opponents. In this case, Zhao was mounting a counterattack against allegations of human rights abuses in China, particularly in Xinjiang.

Instead, it was Prime Minister Scott Morrison who fell into Zhao’s rhetorical trap. Eventually, Morrison did parry back with a message he should have led with in the first place. In a post on his WeChat account (note, this was before it was hijacked), Morrison said that Australia was a ‘free, democratic’ country and was using an ‘honest and transparent process’ to deal with the allegations of war crimes in Afghanistan.

‘Where there are alleged events that have taken place that require action, well we have set up the honest and transparent processes for that to take place. That is what a free, democratic, liberal country does,’ he wrote.

The message was censored on WeChat soon after in a move that still hasn’t been adequately explained by its parent company Tencent, though the article lives on to this day on the Australian embassy’s Weibo account.

The approach Morrison eventually took reflects a way of communicating with the Chinese public that some foreign diplomats working in Beijing call the ‘mirror technique’. It’s a strategy that involves holding a mirror up to yourself and being open and honest about your country’s own shortcomings.

It’s a technique that Ashley Rogers, head of communications at the British embassy in Beijing, has deployed several times in articles that he has written for the embassy’s Chinese social media accounts on democracy, media freedom, Xinjiang and LGBTQ+ issues.

The oblique approach is aimed at creating less heat and more light in an information environment that’s cut off from the rest of the world and primed with a nationalistic ideology. Instead of the comments section filling up with knee-jerk jingoistic responses, the hope is that there’s a chance for genuine discussion with and reflection from the Chinese audience.

‘We know that with our public Chinese audience, they will respond negatively to any direct criticism of the Chinese government,’ Rogers told me via email this week.

‘Therefore, we avoid communications that can be seen as lecturing or finger-pointing. Instead, we aim for a tone that is more conversational and creates space for discussion on these issues. Instead of talking about problems in China, we talk about problems the UK has faced and how we have been able to overcome these and make progress as a society.’

In my 2018 research on Weibo diplomacy and censorship, I discovered posts from the US embassy that prompted reflection from Chinese readers even when it seemed that wasn’t the intention of the post. A January 2018 post from the US embassy informing readers that they wouldn’t be able to continue posting to Weibo and WeChat during a government shutdown went viral, especially after state media gloated about it as yet another sign of American decline.

But even that post had to be censored after online discussion moved on from mere schadenfreude to a serious discussion comparing the US government with China’s own government.

As Rogers pointed out to me, China’s strict censorship regime means that the embassy’s audience is adept at reading between the lines. They don’t need the commentary to explicitly point to issues in China; they can figure it out themselves.

While some (but not many) other embassies commemorated the 33rd anniversary of the Tiananmen Square massacre with posts of flickering candles that were very quickly censored this year, the UK embassy took a different approach, no doubt informed by the backlash it received the last time it posted its own flickering candle.

Instead, the embassy posted an article, penned by Rogers, about the Peterloo massacre which took place in Britain in 1819. The article was censored, but after being live for five hours, Rogers notes—the first time that one of their articles that didn’t refer to China had ever been censored.

‘Within this time, it had been read around 50,000 times (10 times the readership for a normal WeChat article published on our channels) and sparked conversations across social media including on the Reddit: China_irl Chinese channel for Reddit,’ Rogers told me. ‘The comments were overwhelmingly positive.’

When Zhao posted the image of the Australian soldier and the Afghan child, he was sharing a piece of agitprop inspired by testimony uncovered through the Brereton war crimes inquiry. The Chinese Communist Party, by contrast, would never countenance any public inquiry into the Tiananmen Square massacre or, as Australians know too well, an inquiry into the origins of a global pandemic that has wracked the world.

The damning inquiry uncovered something terrible about our country, but it should be something that, unlike the CCP, we’re able to talk about, with humility in a way that shows we’re prepared to hold up a mirror to our own failings. As Rogers reminded me this week, as an open society, this is one of our real strengths and we should play to it.

Serious flaws in EU plan to automate detection of terror material online

The EU’s recently proposed regulation to prevent the online dissemination of terrorist content has sparked significant concern among experts and civil society groups. One of the most controversial elements of the proposal is the requirement for internet service providers to engage in proactive measures to remove or block access to terrorist content on their platforms, including through the use of automated detection systems.

To date, technologies for automatically detecting terrorist content have been seriously flawed. The worry is that the EU’s proposal may drive online platforms to implement these tools despite high levels of inaccuracy. In addition to concerns about endangering freedom of speech or censoring legitimate news, the regulation could pose a major risk to digital evidence of human rights abuses and war crimes.

The spread of camera-equipped smartphones allows for conflicts to be documented in a way which has never before been possible. This footage can be invaluable for human rights advocates, and form the basis for potential war crimes investigations. By their very nature, these images and videos often contain graphic violence, explosions, weapons and other content which is visually very similar to that contained in terrorist propaganda material.

The accuracy of automated tools for content analysis varies widely depending on the type of content. Automatically detecting terrorist content in a video relies on a different set of underlying technologies than detecting written terrorist propaganda, for example. Despite their differences, however, there are a number of weaknesses which virtually all such systems share.

Arguably the greatest and most common weakness is the software’s inability to understand context. An algorithm might be capable of detecting whether a video contains explosions or graphic violence, but may struggle to determine whether that video is terrorist propaganda or a news report. Innocuous phrases like ‘I totally bombed in that meeting’ might be enough to trip up a system based on textual analysis, while euphemistic or even very lightly coded messages pertaining to hate speech like neo-Nazis’ use of triple parentheses, for example, may fly completely under the radar.

A key example of automated detection that all sides in the debate over the EU plan and the broader discussion point to is the ‘hash database’, which internet giants such as Google, Facebook, Microsoft and Twitter use to share hashes (like digital ‘fingerprints’) of terrorist content and, in theory, enables content identified on one platform to be automatically recognised by the others.

While the EU’s proposal singles out the hash database and its close collaboration with Europol as a positive example of industry taking action against online terrorist content, human rights advocates and civil society groups have expressed concern over the database’s lack of transparency or accountability. They point out that there is almost no public information about how ‘terrorist content’ is being defined, how accurate the system actually is in detecting it and what unintended harm the system might be causing.

Automated detection on YouTube, for example, has led to the deletion of thousands of videos  flagged by the Syrian Archive, a civil society organisation dedicated to preserving evidence of human rights abuses in the Syrian conflict. Facebook has been accused of banning, blocking or deleting potential evidence of war crimes, including during the ethnic cleansing campaign conducted against the Rohingya in Myanmar. The destruction of this digital evidence may make prosecution of those responsible much harder.

The EU has attempted to build safeguards into the proposed legislation, including a requirement for hosting providers to offer a complaints mechanism for users who believe their content has been wrongfully removed. Hosts are obliged to store the removed content for six months to allow for complaints or for access by security and law enforcement agencies. This does present a potential window to rescue crucial digital evidence removed by the algorithms—but only if a human being is actively fighting to keep it.

The key point of a redress model based on complaints is that there has to be someone able to make a complaint. In the context of preserving digital evidence of human rights abuses, this presents a fundamental problem. People caught up in violent conflicts or living under authoritarian regimes are busy trying to stay alive and keep themselves and their families safe. It is absurd to think that someone living through the Syrian conflict or coming under attack from Boko Haram has the time, capacity or desire to go through a protracted, bureaucratic complaints process with some distant tech company.

It’s not clear whether a third party such as a human rights group might be able to make a complaint on a person’s behalf. However, even this would require the uploader to realise that their content had been removed, and to have the time and ability to reach out for help to someone less concerned with running for their life.

The sheer amount of content uploaded to the internet every day makes some degree of automated detection of terrorism-related material an inevitability. However, legislators and hosting service providers should think carefully about how to implement such measures—in particular, whether a mechanism which puts the onus on the uploader is really the best way of managing the risk of inaccurate decision-making by an algorithm.

It would be a bitter irony if, in the effort to prevent terrorists from spreading their messages online, authorities and hosting service providers end up destroying the very evidence that could have helped bring them to justice.

Emperor Xi’s censors have no clothes

To watch the Chinese censorship apparatus working in overdrive in the past 48 hours has been to see it reach its reductio ad absurdum in real time.

So sensitive are the proposed changes to the Chinese constitution—which clear the way for President Xi Jinping to stay in power indefinitely—that even arguments made by social media users in favour of the change were caught in the censor’s net. ‘I’m not sure if Germany counts as a democracy,’ hazarded one Weibo user in a tepidly supportive post, ‘but their presidency doesn’t have any term limit.’ The post, like thousands of others, was promptly whacked by Chinese censors.

While some offered opinions, others sprang into action, prompting more action from the censors. Almost immediately after the constitutional announcement, queries for the word ‘emigrate’ spiked on Chinese search engine Baidu. Soon after, the word was blocked online altogether.

By Monday morning, ‘emigrate’ was only the 10th most censored search term on Chinese micro-blogging site Weibo. Outranking it were words one might expect—‘constitution’, ‘amend the constitution’, ‘Xi Jinping’ and ‘ascend to the throne’. But others, more obscure, were also blocked as censors scrambled to stamp out the memes and coded language that Weibo users were creating on the fly in an attempt to circumvent the censorship.

Some of the top words to face the chopping block on Monday morning included ‘Winnie’ from ‘Winnie the Pooh’, who has, since 2013, been used to mock Xi’s appearance. With it banned, censors then had to block references to ‘Disney China’, the company that owns the rights to the famously cuddly bear.

Chinese netizens then began to plumb the depths of both low and high culture to find new allusions—using screenshots as varied as those from popular TV shows featuring characters calling for the death of the emperor to direct references to historical figures like Yuan Shikai, the first president of the Republic of China who proclaimed himself emperor.

As the news spread, the list of banned words blew out to a ridiculous degree, to include ‘I disagree’, ‘boarding a plane’ and the title of George Orwell’s dystopian classic, Animal farm. Even the Roman alphabet letter ‘N’ was blocked.

It would be funny if it weren’t so serious. Behind the gallows humour is growing despair. Those Chinese internet users looking for ways to emigrate surely know in their bones what Jerome Cohen, a Chinese legal expert and New York University professor, wrote in a blog post yesterday: ‘The Chinese Communist Party’s proposed abolition of China’s presidential term limit means that it has forgotten one of the main lessons of Mao’s long despotism. [The two-term limit’s] abolition signals the likelihood of another long period of severe repression.’ He goes on: ‘It will enable [Xi] to move more boldly and increases the risk of his acting arbitrarily and perhaps mistakenly in international relations.’

Aside from the geopolitical risks, it pays to consider this censorship spree as an object lesson in how arbitrary the Chinese Communist Party’s restrictions on free speech can be, and how readily the party can overreach.

Readers may well shrug their shoulders and ask what this has to do with them. So let’s be clear: China’s censorship apparatus is no longer just a boutique concern of China-watchers; it affects all of us.

During the Bennelong by-election in December last year, political parties courted the crucial Chinese–Australian vote in the electorate through the Chinese messaging app WeChat. The discussions that took place on that app between candidates, elected representatives and their constituents were therefore subject to CCP censorship by default.

What if some of those constituents wanted to ask a sensitive question about China’s foreign policy in the South China Sea? What if they wanted to discuss one of the ‘forbidden three Ts’—Tiananmen, Taiwan or Tibet? Would those messages have reached the candidates? How would we know?

We can be sure that views approved by the CCP will be allowed to flourish. While liberal voices have been systematically silenced online, illiberal voices have taken full flight, drowning out discussions domestically and taking aim at foreign targets on the other side of the Great Firewall.

Australian Twitter users got a taste of this treatment in 2016 when thousands of trolls attacked Australian Olympic swimmer Mack Horton and his supporters after Horton called his Chinese rival Sun Yang a ‘drug cheat’.

One by one, big Western companies like Apple, Daimler, Marriot International and Yum Brands are being cowed by hordes of nationalistic trolls for the crime of crossing patriotic red lines. The more abject the apologies, the more the trolls feel empowered and able to move on to bigger targets.

To what extent are our own companies, politicians, journalists and academics already self-censoring for fear of offending Xi’s China?

Money talks in China’s cloistered internet

China’s ambition to shape the future of the internet received a helpful boost last week, courtesy of the CEOs and top executives from the most influential companies on the planet, including Apple, Google and Facebook.

The US tech execs were among the delegates at the highly choreographed World Internet Conference, held in the city of Wuzhen in East China. Their attendance was a coup for Beijing’s censors and won new legitimacy for their vision for the internet in which nation-states are able to wall themselves off from the rest of the world.

China’s famously sardonic social media users were quick to heap scorn on the event, competing to rename it with a more fitting title. Suggestions included the ‘Beggars Conference’, ‘World 404 Conference’, ‘World LAN Conference’ and ‘World Satire Conference’.

Beijing’s zealous censors pounced, ordering the gags to be scrubbed from websites inside the ‘great firewall’—a fact we know thanks to a leaked circular published online by the US-based China Digital Times.

It’s a fitting example of how Beijing’s vision for the internet works in practice. While dissenting voices are censored, those that parrot the official line are amplified.

‘The theme of this conference—developing a digital economy for openness and shared benefits—is a vision we at Apple share’, Apple’s Tim Cook said in remarks that were gleefully reprinted by Chinese state media. ‘We are proud to have worked alongside many of our partners in China to help build a community that will join a common future in cyberspace.’

One headline on the fiercely nationalistic rag the Global Times claimed, with its customary chutzpah, ‘Chinese Net remains open’. Another read, ‘Consensus grows at Internet conference’.

Ironically, Apple removed hundreds of VPN apps from its download store in China at the government’s request earlier this year. But for the tech giant, the moral calculation is easy when you look at the numbers. Speaking at the conference on Sunday, Cook said that Apple’s App Store has earned nearly $17 billion for 1.8 million Chinese developers, who together account for about one-quarter of all developers’ revenue through the store.

The Chinese market is proving to be alluring in new ways. As new technological fields like AI open up, the country seems at a unique advantage due to its huge population and a lack of concern about privacy.

It’s enough to have even Google—which famously retreated from the mainland Chinese market in 2010 on a matter of principle after it declined to censor its results—outlining big plans to return to the market in a big way.

Alibaba executive chairman Jack Ma offered a blunt warning to the Western tech companies that have been agonising over how to engage with the Chinese market. ‘People complain too much’, he said at Wuzhen. ‘[Foreign companies] are determined to come. Follow the rules and laws and if you’re unhappy, leave’, he added. ‘This is not a market [where] you can come and go.’

Beijing has been making slow, steady gains in promoting its vision for the internet, which it has dubbed ‘cyber sovereignty’ in multiple forums. The strategy came into sharper focus at Wuzhen after the government unveiled its plans for a ‘Digital Silk Road’ that would run along the Belt and Road Initiative. The developing world, in particular, is being wired by China, making the prospect of a global ‘Splinternet’ a worrying possibility.

‘China stands ready to develop new rules and systems of internet governance to serve all parties and counteract current imbalances’, Wang Huning, a member of China’s top decision-making body, the Politburo Standing Committee, told an audience at Wuzhen that included Cook and Ma.

While Wang didn’t shy away from spruiking the government’s concept of ‘cyber sovereignty’, it was the delegates to the PLA’s Cyberspace Strategy Forum—held in Beijing a week before the Wuzhen conference, and to an entirely local audience—who put it more bluntly.

At that event, China Institute of Contemporary International Relations’ Zhao Chen spoke about the spirit of ‘deepening comprehensive governance, widening the road to an Internet powerhouse’, and quoted Xi Jinping:

Currently, the rivalry among major powers in cybersecurity is not just one for technology, but rivalry for ideology, for the power of discourse … We should safeguard our cyberspace sovereignty with righteousness, and make our voice loud and clear.

For the Chinese government, cybersecurity isn’t only about hackers and cybercrime; it’s about ‘information security’—code for ensuring liberal foreign ideas like democracy and the separation of powers don’t gain a foothold inside the middle kingdom.

As with each previous Wuzhen conference, the internet blocks on Facebook, Google, YouTube and Twitter were removed for the benefit of the international visitors. Perhaps next year Beijing will be confident enough to leave them blocked and let its visitors experience what ‘cyber sovereignty’ means in practice.

How far do the tentacles of China’s censorship reach?

Foreign government efforts to undermine Australian democracy have been making headlines in the past few weeks, but we’re missing things.

In announcing new legislation to ban foreign political donations, Prime Minister Turnbull told us: ‘Foreign powers are making unprecedented and increasingly sophisticated attempts to influence the political process, both here and abroad.’

The 2017 foreign policy white paper signalled a protracted battle with foreign interference, stating: ‘The Government is concerned about growing attempts by foreign governments or their proxies to exert inappropriate influence on and to undermine Australia’s sovereign institutions and decision-making.’

And then, of course, there have been the shady examples of influence-buying.

While the recent focus has been on political donations and ‘how China’s spies operate in Australia’, one area of interference in the cyber domain has been overlooked: how Beijing can exert influence through the penetration of Chinese social media platforms in the Australian market.

A 2017 Deloitte survey suggests about 1 in 7 Australians source their news from social media. For around 1.5 million Chinese-speaking people in Australia, that includes the Chinese social media platform WeChat. That’s a problem because WeChat takes direction from the Chinese Communist Party (CCP), which means Australians who source their news through WeChat are having their news diet dictated by a foreign authoritarian state.

Research conducted by the Citizen Lab at the University of Toronto has documented the extensive but subtle censorship imposed on WeChat users by the Chinese authorities. That includes blocking messages that contain censored keywords (such as Tiananmen massacre or Radio Free Asia), links to banned websites (including news sites critical of the Chinese government) and images with banned political content (such as images associated with controversial events like human rights crackdowns).

This censorship can be tricky for users to detect. For example, when a user sends a message with a censored keyword, they are not advised that their message was blocked; the message just doesn’t appear on the receiver’s end. The censorship regime is also far stricter in group chats than in private messages, suggesting an effort by the authorities to prevent mass dissemination of messages potentially damaging to the CCP.

Through rigorous testing, the Citizen Lab was able to demonstrate that users who originally sign up with a mainland Chinese mobile number, then later switch to an international number, remain under the same censorship as they would back in China. This has the potential to sweep up Australian holidaymakers, migrants of Chinese descent (more than half a million Australians were born in China), students, academics and business people. While users who establish a new account from Australia can avoid keyword censorship on WeChat, they still have some websites blocked and are subject to any future changes imposed by the CCP as it continues to tighten censorship at home.

This reach into Australian society should be of concern to Australia. WeChat has several potential pernicious applications. It is being used to limit the material Chinese Australians can read about issues that the CCP deems sensitive. It could also be used as a vehicle for monitoring dissident students in Australia who use the platform.

This is worrying because, as the white paper tells us, by 2030 in purchasing power parity terms, China’s economy will be nearly twice that of the United States, and China is already challenging the US for dominance in our region. As Hugh White outlines in his new Quarterly Essay, China is likely to use that power to push us into some uncomfortable territory. Its ability to shape Australian debate by exerting control over a major media channel like WeChat is one important way it could influence outcomes in Australia.

This has the potential to erode one of our key national strengths. As White also observes, in confronting the challenge of China’s rise ‘we have some real and growing assets, including over 1 million Australians of Chinese descent. It is too easy to overlook the vital and obvious contribution they will make to helping us find our way in a Chinese-dominated East Asia.’

There’s nothing wrong with Australians choosing to use WeChat, and all Australians are fortunate enough to have a diverse range of news sources they can access. But we deserve to know what is and isn’t being censored from our news diet, especially when it comes to dominant market platforms. Australia can’t stop the CCP from censoring WeChat, but it could require that all social media companies publish their censorship rules as a condition of operating in Australia (as companies like Facebook already do). That would at least allow us all to see what we’re missing out on and make it easier to pressure for change.

Internet censorship: how China does it

Last month, Chinese state media published articles commemorating the 30th anniversary of China’s first-ever email: ‘Across the Great Wall, we can reach every corner in the world.’

The email was sent from a research institute under China North Industries Group Corporation in Beijing on 14 September 1987 and received by the University of Karlsruhe in Germany at 8.55 pm on 20 September 1987.

Techno-optimists believed that the internet would ensure a free flow of information and ultimately a democratic society in authoritarian states like China. Thirty years on, however, China has instead built a Great Firewall, a vast hardware and software system that aims to prevent access to undesirable websites and censors sensitive content.

While China’s extensive internet censorship isn’t new, there are several myths about how information controls are actually enforced by the Communist Party of China (CPC).

As of June 2017, China’s internet population had reached 751 million, which is more than the total population of Europe. The Chinese social media landscape is complex and vibrant. Tencent’s WeChat, China’s most popular instant messaging application, has 889 million monthly active users. As of April 2017, Sina Weibo, China’s Twitter-like service, had 340 million monthly active users, generating more profits than its counterpart in the West. And the Chinese internet is full of other booming industries, such as the gaming sector, bulletin boards and live-streaming platforms.

Those statistics help to explain why the CPC increasingly sees the internet as the core battleground for ideological control and its very own survival. Hence, we’ve witnessed multiple clampdowns on VPNs (virtual private networks, which help internet users jump over the Great Firewall and evade censorship), efforts to control narratives in the ideological sphere through the so-called Document 9 and other directives, and restrictions on video-streaming services as well as chat applications, all under the rationale of ‘preventing the spread of illegal information’.

Successful implementation of censorship, which often overlaps with the state’s propaganda agenda, is also a demonstration of the government’s strength in social control. As political scientist Haifeng Huang pointed out, it’s one of the straightforward tools that’s used to remind people of who’s in charge.

A widely shared perception is that the Chinese government can effectively censor unwanted content in a timely and monolithic fashion. That’s only true to some extent. Analysts tend to overestimate the state’s capacity to impose uniform censorship while underestimating the policy implementation hurdles faced by central and provincial bureaucratic agencies and the extent of the state’s reliance on its symbiotic relationship with internet companies in China.

China’s internet bureaucracy is a two-tiered system. Censorship directives come either from the central government or the provincial one, while high-level policies are made centrally. The management of internet companies follows the principle of territorial management (属地原则, shudi yuanze).

For example, a decision to remove content from WeChat should come from Guangdong Provincial Cyberspace Administration since Tencent, WeChat’s parent company, is based in Guangdong. In contrast, general regulations that apply to all internet platforms are drafted by the central Cyberspace Administration of China (CAC).

Below is a list of regulations released by the CAC in a two-week span between August and September 2017.

[table id=1 /]

The implementation of censorship often involves pushing the burden of content monitoring and controls down to the lowest level possible. All internet companies operating in China are subject to laws and regulations that hold them legally responsible for content on their platforms. They are expected to invest in staff and filtering technologies to moderate content and stay in compliance with government regulations. Failure to comply can lead to fines or revocation of operating licences.

While the state’s reliance on companies’ compliance serves as an effective sword of Damocles, it can result in ‘cracks in the Great Firewall’, if it’s in companies’ interests to resist government efforts and provide otherwise censored content to compete with other platforms. It makes a temporary failure of censorship, whether due to technical glitches or deliberate defiance, possible.

Moreover, there is no clear-cut, easy-to-follow guideline for companies to decide what ought to be filtered from their platforms. In 2010, China’s State Council Information Office published a major government-issued document on its internet policy. It includes a list of prohibited topics that are vaguely defined, including ‘disrupting social order and stability’ and ‘damaging state honour and interests’. This regulatory environment pushes companies to over-censor, a phenomenon China expert Perry Link described as ‘the anaconda in the chandelier’.

The commercialised process often leads to a decentralised and fragmented application of censorship on the Chinese internet. Those amorphous censorship practices have also caused unintended abuses, including China’s ‘black PR’ business and companies’ blocking of non-sensitive content out of commercial interest.

Studies of mobile gameschat applicationsblogssearch engines and live-streaming platforms in China by the University of Toronto’s Citizen Lab and others have consistently found variances in how companies implement censorship. Those empirical studies reveal that while companies may be receiving general directives on prohibited content, they haven’t been given a central list of banned keywords.

Chinese leaders are well aware of the dilemmas in the country’s current internet management system and its management of content via companies. That is why Xi’s administration has taken big steps in reshaping China’s internet control agencies by establishing the CAC and passing the influential Cybersecurity Law.

It’s unclear how effective those internet management reforms will be, but it seems evident that 30 years after it first embraced global cyberspace, the CPC has decided that it’s in the party’s best interests to ‘traverse the hurdle represented by the internet’ rather than simply letting its people reach ‘every corner of the world’ online.