China has been regulating artificial intelligence for years, challenging the assumption that technology is evolving too rapidly to regulate. As concerns grow over tech company accountability, what lessons (if any) can Australia and other democracies take from China?
In this episode of Tech Mirror, Johanna Weaver speaks with Professor Madeline Carr, Professor of Global Politics and Cybersecurity at University College London and visiting chair of technology at Schwartzman’s College at Tsinghua University in Beijing.
Together they explore China’s evolving approach to AI regulation. From recommender algorithms and synthetic content to generative AI and safety frameworks, Madeline shares insights from on the ground in Beijing. They unpack the differences between US and Chinese approaches to innovation, the importance of liability in holding technology companies accountable, and consider how middle powers could govern foreign technologies in a world where the best innovative tech isn’t necessarily coming from a major ally.
Links:
US tech leaders are urgently calling for rules on AI – China already has them | The Conversation
We Must Pace the Frontier | Dario Amodei
Algorithmic Recommendation Regulation (Order No. 9) | Cyberspace Administration of China (CAC)
Internet Information Synthesis Regulation | Cyberspace Administration of China (CAC)
AI Safety Governance Framework 3.0 (mandarin)
Trsnscript
Johanna Weaver: [00:00:00] The Tech Policy Design Institute acknowledges and pays our respects to all First Nations people. We recognize and celebrate that among many things, Indigenous people were Australia’s first tech innovators.
Madeline Carr: One of the things I think is so important to understand about Chinese tech policymaking is the power of the process here.
Guidance, recommendations, all kinds of policy instruments in an incredibly agile, flexible, fast way. It’s about thinking what are the problems that we have? Lifting 800 million people out of, out of poverty over the last 25 years is… That’s a big problem, and there was a lot of technology that went into, to addressing that problem.
They don’t use that language of a race with the United States.[00:01:00]
Johanna Weaver: Hello, and welcome to another episode of Tech Mirror. My name is Johanna Weaver, and I’m your host. This is the podcast where we talk about how humans are shaping technology and how we can make sure that it’s shaping a world that we all want to live in. And today my guest is Madeline Carr. Madeline is an Australian, importantly, but she is also a professor of global politics and cybersecurity at the University College London.
And Madeline was one of the first people that I encountered when I started working in this field that was looking at the intersection of emerging technologies, cybersecurity, and geopolitics. So Madeline is actually in part responsible for my career. It’s wonderful to have you on the pod today, Madeline.
Madeline Carr: Thank you so much, Johanna, and wonderful to learn that I had any, any part to play whatsoever in spawning the great Tech Policy Design Institute.
Johanna Weaver: [00:02:00] Ah, yes. Well, I mean, it was definitely, um… you were looking at these issues before they were cool. That is definitely fair to say. And you’re currently in Beijing as the visiting chair of technology at Schwarzman’s College at Tsinghua University in Beijing, which for those who are not super familiar with Chinese universities, it’s kind of the equivalent of Harvard University in the US.
So it’s, like, the most prestigious law university. What led me to reach out to you, Madeline, was an article that you had published in The Conversation about US tech leaders are urgently calling for rules for AI, but China already has them. And so we’ve invited you on today to talk about that article, to look at what China is doing in terms of regulating AI, what we should be learning from that, what we shouldn’t be replicating, uh, in the Chinese model.
We’ll have a look from a geopolitical perspective and compare what the US and China are doing, and probably also touch on the concept of liability as well. [00:03:00] But before we delve down into that level of detail, Madeline, can you just give us a bit of context in terms of… You’re recording this from Beijing.
So what are you working on at the moment? How, how have you come to be sitting in Beijing on this podcast?
Madeline Carr: Yes. How indeed. I mean, I guess I’ve been coming to China over the last decade or more for track 1.5 and track two dialogues between the UK and China on cybersecurity and cybercrime. But a few years ago, I was approached to join Schwarzman College as the, as a visiting chair, and that has been just a really incredible opportunity for me because I spend sort of three or four months of every year here teaching in the program, but also participating in talks and forums and meeting Chinese colleagues.
And, you know, for someone who’s interested in tech policy, and [00:04:00] particularly Chinese tech policy, it’s just so, so helpful to be here to get a deeper understanding of how Chinese people and the Chinese government approaches technology, how they approach technology policy And how they bring together their kind of objectives and ambitions into governance frameworks, because for you and I, it’s that governance piece that is so, so powerful and so important.
And yeah, it’s been, it’s been an extraordinary, uh, experience and privilege, I would say, to be able to spend so much time here understanding those things from a Chinese perspective.
Johanna Weaver: And I guess one thing that I always find fascinating is reflecting on the atmospherics of being in Beijing at the moment.
There still aren’t a lot of Australians in Beijing traveling. There are definitely more than there were a few years ago, but can you just help paint a picture of what it’s like on the ground in [00:05:00] Beijing right now, how technology is sort of interacting with day-to-day life, um, before we go– get into the nerdy regulation part?
Madeline Carr: Yeah, of course. I’m very fond of Beijing because I spend a lot of time here. It’s true that there aren’t a ton of Westerners here, but, uh, you know, there are quite a few Western students at the universities on exchange programs, and I think less than there were before COVID. But one of the things that has made a big difference in China over the last year is, uh, they have visa-free travel for a lot of countries.
So where it used to be quite painful to apply even for a tourist visa to China, you don’t have to anymore. There’s a visa waiver system in place, and I think that’s made a big difference to the volume of people that come on a holiday to, to visit this place. In terms of just day-to-day interaction with technology, I mean, I just came back from a, a visit to quite a rural area in the southwest of China, [00:06:00] Yunnan Province, and even in, in a place like that, you know, if, if I wanted to buy a, a, you know, a banana from someone on, on a streetside stall, I could pay with, with my payment app.
So the places… I- i- it’s incredibly integrated and wired up and connected for this kind of extreme level of convenience and sort of seamlessness that makes it… Yeah, it’s a very easy place in a strange way, ’cause the language is difficult for Westerners, but it’s a very easy place. Once you have those elements in place, it’s very easy to travel.
And a, a colleague at dinner last night was telling me that she, uh, was out for dinner with a friend, and he offered to give her a lift home. And he said, “I have a car. I’ll, I’ll call my car.” And he called it. This was in Beijing, and it was a driverless car that pulled up to, to take them home from dinner, which she was, you know, wildly excited about.
Yeah, and he told her that car cost about $20,000 US, so [00:07:00] There’s a lot of optimism about technology here that I don’t see at home in the UK or in the West generally. There’s much more optimism about it- Hmm … much more excitement about what it will do to improve the human condition rather than degrade it.
Johanna Weaver: And is there concern about, you know, when, when we think about technology and China, people often think, “Okay, yes, it gives that convenience, but it’s also the sort of surveillance and monitoring that comes with it.” Is that something that is there sort of on the surface? Is it overt, covert? How, how much does that play into conversations?
Madeline Carr: I can’t sort of speak for the 1.5 billion people here, but I don’t encounter that in my day-to-day exchanges with people. They don’t have that anxiety about surveillance, and it was really eye-opening when I came here because, of course, that’s something that we are very alive to and sensitive about. I don’t see that [00:08:00] sensitivity here, and on the contrary, what I’ve come to recognize over the last four years or so of being embedded in this system, even if temporarily and sporadically, is that particularly when you look at issues like cyber crime, those things are much less of a problem.
They don’t have burner phones here, for example. If you want a phone, you have to show your ID, and that phone is linked to your ID and to your bank account. The flip side of that is, yes, the, there’s not a lot of, uh, you know, you can’t do things secretly. That’s true. But the flip side of that is that there’s a kind of high level of personal accountability.
So if something goes wrong, if, for example, if I, if I should buy something in a shop and I see they have charged me twice, they would just… If th- they would reverse that themselves when they saw it because th- there would be no escaping that. You know, everything is traceable. Everything is [00:09:00] trackable, and from a kind of privacy perspective, we wouldn’t probably tolerate that.
But if you’re focusing on security and, and- Hmm … crime prevention, it, we sort of in some ways look longingly, you know, at, at what can be done to prevent those kind of online harms. Things like bullying, online bullying and stuff, absolutely can’t happen because you have to put your name on your social media accounts, and yeah, there’s kind of something to be said for that.
I’m not suggesting- Yeah … we could transplant that to Australia or the UK, not at all. But it does show you the other side of it.
Johanna Weaver: Maybe let’s talk about some of the laws first. So the article that you wrote took the view of following how China has progressively regulated [00:10:00] artificial intelligence, not necessarily through, you know, one overarching piece of regulation, but progressively as the technology has evolved, and kind of challenging this assumption that the technology is evolving too fast for regulation to keep up, because actually China has progressively kept up.
So can you take us through some of the… a bit of a timeline in terms of the key pieces of regulation or legislation that is governing artificial intelligence in China, and what the particular parts that they are, are focusing on? But maybe first if you can comment on that sequencing piece and the challenging that it’s not possible to regulate and to keep up.
Madeline Carr: Yeah, absolutely. I mean, I wrote that, that article out of frustration with this kind of global discourse that emerged after Dario Amodei’s post. Which, you know, was basically premised on this notion that governments have to do something, and they have to do something quickly to stop this, [00:11:00] uh, you know, out of control, uh, sector from causing harm.
And having spent time here studying those Chinese instruments, I felt that there was a real lack of understanding of how we could say the other big AI superpower is approaching those challenges and how they have done this incrementally. And, and, you know, I’ll, I’ll run through that, that sort of sequence of, you know, major pieces.
But one of the things I think is so important to understand about Chinese tech policymaking is the power of the process here. And it- it’s almost– I would say it’s, um, asymmetric advantage that China is able to draft kind of guidance, recommendations, all kinds of policy instruments in an incredibly agile, flexible, fast way.
They’re [00:12:00] very innovative, not just in terms of technology, but in terms of technology policy. They’re very innovative in the way they experiment, in the way they will reverse things that don’t work out. It’s really extraordinary to see what can be done with, you know, the right, uh, mechanisms in place. And again, I’m not suggesting that, that– the way they do that can be picked up and transplanted to another place, ’cause there are all kinds of structural reasons why they can do that.
But I do think there are some lessons to be learned or at least to be examined of how China does this. So basically, back in twenty twenty-two, they developed these provisions on algorithmic recommendations, and this kind of preceded AI, but it links into AI. So they wanted to ensure that users had some sense of choice in the [00:13:00] algorithmic recommendations that were presented to them, and that there was also a level of transparency around this and accountability for platforms and service providers in how those algorithmic recommendations were being developed and, and presented.
A year later, basically a year later in 2023, they developed these provisions on deep synthesis internet information services they call it, and this basically covered synthetic or AI-generated text. That was really important because we see this rise of synthetically generated or AI-generated images and audio and video.
These things are very fundamental risks to cybersecurity, and they’re also risks to personal security. So they have, in addition to providing that all AI-generated data is clearly labeled as AI-generated, they also took steps around the same time to ensure that anyone [00:14:00] online offering advice, medical advice or financial advice, for example, has to be fully qualified.
So you can’t just now go, go on, you know, TikTok or whatever and, and give investment advice and what’s worked for you in growing your portfolio if you’re not a financial advisor, and same with medical advice. So that was part of a… You could say validating of online information that people were, were absorbing and, and engaging with.
Um, a few months later in, in August 2023, they developed these interim measures for the management of generative artificial intelligence services. And there they, they basically address things like training data. So we know that the data that’s used to train models is very important, and you hear this often from African countries, for example, you know, pointing out that the models that they’re– are being made available to them are not [00:15:00] trained on relevant data for them, and hence they’re not particularly useful.
But they also, they also need to respect intellectual property and which of course has been a huge, a huge debate in, in the West, and particularly in the UK, where artists have, have taken exception to their work being used in training these models. But then just most recently there was a, the latest update of the AI s- safety governance framework, and this is, I guess, the most– to me, the most interesting document because it really lays out how the Chinese government is thinking about this, how it’s approaching it, and they’re, they’re up to version three of this, just came out a few weeks ago.
First of all, it classifies a whole lot of risks that are being iteratively understood or incrementally understood, and it also lays out what’s very interesting for people like you and me. It lays out not only technical mitigations against these risks, but it lays out governance mechanisms. It [00:16:00] lays out proposals for them.
So yeah, I think there’s… You know, that, that’s kind of the trajectory that I outlined in the piece and that I’ve, I’ve observed here.
Johanna Weaver: And we’ll put a link to the piece in the show notes as well as to some of these pieces, some of the English translations to some of these instruments, as you refer to them.
But it is quite extraordinary to reflect on that, right? So we’re, in Australia, focused on digital duty of care at the moment. A big focus on that is fix our feeds and the algorithms, which in my view is important but way too narrow. We need to be looking at it in a much more expansive way. But the Chinese approach back in 2022 had that, um, had that covered.
And then, you know, the d- conversation around synthetic content, deepfakes, et cetera, is one that’s always ongoing and continues to be ongoing here in Australia, again, having dealt with that years ago. Can you… I’m really intrigued and interested by, um, some of the substance of these proposals, but also your comment about the process.
Because a lot of these are published [00:17:00] as, as rules or provisional draft instruments, interim measures, guidelines. Is there anything in that process, appreciating that it is, sits within a much larger, um, Chinese bureaucracy, are there parts of that that we should be looking to transplant into Australian or UK structures?
And given that you’ve got that familiarity with, with both sets of systems, is it possible to bring some of that innovation, do you think, to the way that Australia’s considering regulating, for example?
Madeline Carr: Yes, I do think so, and I think when you look at some of the, especially the technical mitigations in the latest update of the framework, for example, the proposal that AI agents be given a unique identifier.
I’ve heard proposals along the lines of can they be assigned an IP address or some other unique [00:18:00] identifier, and I think something like that could really be quite game-changing because- You know, getting back to the, what we were saying earlier about the relative merits of not being able to buy a burner phone and u- and use it to commit crime.
Being able to trace an agent and know where it has been, I mean, this is pertinent for Australia, right, for the recent Medicare i- intrusion. Being able to know and identify where has that agent come from and where has that been, I think is gonna be pretty fundamental. And I think there’s real value in, in considering how we can collaborate with the Chinese as they develop that technology to, to do something like that.
There’s no reason why Australia needs to just wait and kind of accept that from another country, but I think there’s real potential and value in looking at… We all have that interest.
Johanna Weaver: Mm. Yeah.
Madeline Carr: We all have that [00:19:00] interest in being able to identify agents online. How can we do that together?
Johanna Weaver: So we’ve, we’ve spoken there through the, I guess, the timeline of how China has approached the regulation.
We’ve spoken a little bit about the processes that they use, and also their willingness to say, “This doesn’t work, and we need to do something else,” which I think is actually quite unique. There’s not a lot of governments that have that, and that’s partly because they don’t have a opposition that’s gonna say, “You got it wrong,” so let’s let’s acknowledge that.
Madeline Carr: That’s
Johanna Weaver: right. That’s right. Um, but what do you think us, countries like Australia should learn about regulating iteratively based on the Chinese approach? Is that the key lesson out of here? When you wrote that article, what was the sort of key takeout that you wanted people to have from this? Is it the iterative nature?
Is it that it’s possible to do it? Like, what’s the focus?
Madeline Carr: I think there are a couple of takeaways from what I’m seeing here. One is that we can’t [00:20:00] pick up wholesale and replicate what China does, and even China would say that. China is n- not, it’s not proselytizing or evangelistic about its, its approach to policy.
It never really- Seeks to impose that anywhere else because they, they, they honestly don’t believe that it’s possible for anyone else to do what they do in China. And it is certainly the fact that there’s not an opposition to answer to when things don’t work out. It does give them freedom to turn around on things that haven’t worked or to, yeah, issue correctives when policy experiments don’t pan out the way they expect them to.
But I think that there’s value in looking closely at individual instruments and the, the, the clauses and proposals within those instruments to see, well, what could we do? For example, the unique [00:21:00] identifiers for agents. That’s not a Chinese problem, and it’s not a Chinese solution. That is a global problem, and that’s something that everyone would have a vested interest in understanding.
I don’t think you would find many policymakers who would argue for let’s, let’s allow these agents to just roam through our systems anonymously and do as they will. I mean, another proposal is that those agents must be programmed to comply with relevant laws, and why they haven’t been is a good question, particularly the proprietary models that, that are coming out of America with open source models coming out of China, that’s– becomes more of an individual problem.
But I think, yeah, looking at those instruments very carefully and considering what could be useful What might we be able to adapt, not in, in whole, maybe in part? I mean, what I’m still trying to get to grips with here, and when I, when I have, I’ll check back [00:22:00] in with you, is how do they do this so flexibly and so agilely?
Leaving it… leaving aside the kind of structure of, of their governance system here, but how do they do that? And I know that they have slightly different relationships with think tanks, with academics. There’s a more, let’s say, a more joined-up line of communication and research from researchers in universities, industry think tanks, and what the government needs, and this is something that we’ve worked hard on in the UK to try and bring a bit more synthesis and coherence to those conversations, and we, we continue to struggle with it as well, though.
How do you get that information out of academic conference papers and journal articles and into the hands of people who need it? It doesn’t happen as smoothly as it needs to in the UK. I bet it’s [00:23:00] similar in Australia, and somehow here they seem to be able to get what they need, and I think there’s something to understand there about how do they extract that support from the rest of their research community and infrastructure.
Johanna Weaver: And I think, um, the, the research community, but also the relationships with industry as well, and that sort of really close connection, which inevitably is what is, what enables the government to be able to understand w- you know, where the frontier is that they need to be regulating in a way that just doesn’t exist with the separation between Silicon Valley and the rest of the world, perhaps r- not even Silicon Valley and Washington.
That’s an interesting segue, perhaps, to think about the different ways that China and the US think about global governance of AI. So China [00:24:00] recently established the Shanghai World AI Cooperation Organization. Um- That’s right … when you reflect on the way that China’s approaching engaging globally, taking your point that they’re not proselytizing the particular method of governance process-wise, but they are quite proactive in saying, “We are being responsible.
Our AI is being built in a way that can be controlled and can be regulated.” What’s your reflection on that contrast between that and the US?
Madeline Carr: Yeah. I, I mean, this is quite fundamental to different approaches to global governance and to the rest of world/Global South, how- however we, we would, would think about it.
I guess in, in America, the last, what, maybe we could say 70 years or more, has been driven by a particular [00:25:00] belief in innovation, which is that government should play very little role. They should l- you know, stay out of the way, leave it to the private sector, and that’s the best way to propel innovation.
And then the, the benefits of what happens in the private sector will trickle down, either financially or in, in other ways, without a great deal of direction, that there’s this kind of market drivers for innovation that will benefit everyone. That’s not really what you see here, and- Here, I think the approach is much more about problem-solving and using technology for problem-solving.
So what is the problem? The problem is our energy supply. So we need, we need technology to manage the energy grid. What is another problem? Um, the [00:26:00] manufacturing base. As China’s middle class grows, the manufacturing base looks perhaps at risk of moving to countries like Vietnam. What to do? Well, advanced robotics, you know, advanced manufacturing, th-these types of things.
And I think for China, it, that’s a fundamental difference. It’s not just about making money and then hoping that will trickle down to everyone else. It’s about thinking, what are the problems that we have? You know, lifting eight hundred million people out of, out of poverty over the last, whatever it’s been, twenty, twenty-five years is, is…
That’s a big problem. And there was a lot of technology that went into, to addressing that problem, and, and that continues to be the thinking. So now, you know, in twenty twenty-six, as China I guess, is on the, the doorstep of this new global leadership role that is very much [00:27:00] structured around supporting the Global South and supporting the developing world and those countries that are still coming up behind it to escape poverty.
Now, you can of course see that somewhat cynically or instrumentally, as Kishore Mahbubani points out, the, the former foreign minister of, um, Singapore. He’s, he, you know, he’s, he’s famous for this line. He says, “Only 13% of the world is in the West, and that 13% has a particular view of China. The other 87% has quite a different view of, of engagement with China.”
And China also understands that. So you could see that quite cynically and instrumentally that this is a good profile of global governance, but you could also see it as, as, as that’s their approach. They would say that they feel a responsibility to support those countries that are coming up behind them, and they also…
I mean, this is one of the interesting things about the global [00:28:00] governance work they’re doing. They have a great, a great institute set up at Fudan University, and I’ve engaged with those, th- those scholars quite a bit, and they, and they pointed out that people in the Global South, when they hear this discourse about AI safety, they really respond negatively to that because for them, it sounds a lot like, um, like climate change discourse.
Now we’ve industrialized, we’d like everyone to be more, more conscious of, of environmental degradation while they’re still, um, struggling to kind of get their economies and their, their social systems off the ground. So for them, it can be received in the Global South as the West again, the developed wealthy West again saying, “Be careful now.
D- let’s not go too far and too fast. Just slow down with, with AI.” And that that’s not received all that [00:29:00] well in, in the rest of the world.
Johanna Weaver: And I think it’s when you combine that with what you had said before about the focus of Chinese AI on problem-solving, and then you combine it with that history of, “Don’t tell us to slow down, now you have the advantage,” it is very powerful.
And I, I attended a conference in India a couple of years ago and came away just so struck by- Mm … how every Western person was talking about the risks, the dangers, and everyone from the rest of the world was talking about the opportunity. And even if you look at, you know, the AI Impact Summit held by India versus the AI safety summits that have been held in the UK or France, um, Seoul, it’s such a different focus.
In the conversations that you’re having in China, is there a conceptualization of being in a race with the US? You know, ’cause that’s how the US is always perceiving this is, right, “We have to beat China. We have to be in that [00:30:00] race.” Yeah. How is, how is that conversation unfold?
Madeline Carr: I never hear that here, and that’s not to say it never happens, but I never hear it.
They don’t use that language of a race with the United States. It may be surprising the extent to which China still admires and, and respects the United States, and particularly big tech companies and great universities. The other thing I never hear here, in addition to racing to the, to, to the US, I don’t hear them talk about being number one.
And in fact, there’s a kind of saying in, in Chinese where basically it’s better to be number two. And I see that reflected a lot in China. Number one is a dangerous position. Number two is, is a safer position, and it’s still great, and that’s… That, that sort of resonates with me a lot when I [00:31:00] hear Chinese people talk about their relationship with the US, the tech competition with the US.
W- it, it gets back to that, that thing of problem-solving. So they have their own set of problems that they’re directing AI towards, and that’s what they’re focused on. So if America does great as well, it’s fine. The thing that they do object to, of course, is the export bans on things like chips and chip fab equipment and that type of thing.
Of course, they object to that, and they feel that’s a very kind of blunt instrument to try and slow China’s progress. But I don’t hear that race… It just doesn’t come up. They’re very focused on what they’re doing, and they don’t like the US getting in the way, but yeah. That’s how it seems to me.
Johanna Weaver: There’s the recent, um, meeting between the Chinese president, the US president, and putting in place the, you know, startings of conversations around emergency communications during a [00:32:00] s- a AI crisis, for example.
Is that something that is also perceived as posturing, or is that seen as necessary leadership?
Madeline Carr: I mean, I think it’s, it’s seen as, as necessary and desirable. I think, I think with any kind of potential global crisis or anything that, that has the potential to escalate, there’s a, you know, generally a preference for good communication and communication channels.
The th- the kind of feedback that I got about the recent meeting in the US and the, the earlier one, earlier in the year, was that… And this was not from Chinese colleagues, actually. It was from, it was from American colleagues. W- well, there was a frustration on the Chinese side that there was insufficient preparation for those meetings in both cases, that there was a lack of preparatory meetings.
There was a, a lack of discussion about what can, you know, what could be possible. And [00:33:00] American colleagues pointed out to me that this is partly a consequence of the, the absence of professional diplomats in the State Department. Mm.
Johanna Weaver: Yeah.
Madeline Carr: So without that real professional class of experienced diplomats that know how to prepare for these meetings, that know how to get things in place, that know how to lay the pathway for something substantial to come out, then nothing comes out, and I think that’s, that was a real frustration on the Chinese side, that there just wasn’t this- preparatory work to make the most of those, those two meetings.
But not to say that they weren’t… Look, they’re still useful, and it’s good to see that those, those leaders can meet and… But it was frustrating that there wasn’t more of substance that, that happened during those meetings.
Johanna Weaver: One thing that I’ve found myself saying a lot recently, actually, in respect to copyright, but applies [00:34:00] here as well, is that negotiation isn’t magic, right?
It requires- Mm … a huge amount of hard work and preparation to really- Exactly … understand the other person’s perspective and where the commonality is. And then, you know, to actually get sensible people in a room to have the conversation, to make the concessions, because there’s always concessions in that type of- Exactly
um, conversation. Um- Yeah … and, yeah, I mean, the hollowing out of the State Department is having huge implications across every domain. Think about- Yeah … development assistance or engagement bilaterally with different countries. Um, it really, it’s not something that I think people focus on, but you’re absolutely right in terms of that, that impact in the ability to drive and to get good outcomes.
So, you close your article talking about liability and the fact that, you know, if the US continues perhaps to take an approach that is sort of hands-off, let it rip, let the companies do what they like, despite the fact that the companies are calling for this regulation, although sometimes we take that with a grain of salt, too, [00:35:00] because we’ve heard that before, that liability might be a way that the world globally responds and puts in place a regime that regardless of what the US government may or may not wanna do, holds companies accountable.
Do you see something like that happening? There is that liability that exists for companies. Is it possible, do you think, to have, um, that type of regime, and would it be something that you think we would need the US government participation in? Is it something that a middle power like Australia could build a coalition of countries to sort of implement something like this?
This concept of liability is one I’m focused on a lot at the moment.
Madeline Carr: Okay. Yeah. N- not surprising. Um, I guess from years of working in cybersecurity, I’ve felt that essentially liability really sharpens the mind, doesn’t it? And i- i- in some ways- Changes the incentives. That’s actually the reality. Yeah It changes the [00:36:00] incentives.
That’s the reality, Johanna. Australia’s produced this fabulous guidance for corporate governance of cyber. We looked at a lot of these because we developed one in the UK as well, and the Australian one is just incredibly sophisticated, valuable document in my mind, and essentially What we’re seeing in countries– I don’t know if, if this is the next step in, in Australia, but in the UK, there’s a discussion, now that we’ve issued this voluntary guidance, that in some not too distant future, that this will be written into the corporate code.
In other words, directors of companies will become liable for the cybersecurity of their organizations, not just fiscally liable, liable for the cybersecurity as well. And this i- is at the end of decades of trying to get business leaders to apply themselves more enthusiastically and energetically to cybersecurity, and realizing at the end of the day, [00:37:00] there are not other incentives that are working.
There are not financial incentives. Business leaders are very good at evaluating risk and costing that risk, and that’s what they do. And so I think v– it’s, it’s really from that experience in cybersecurity of seeing governments struggle for years to encourage, plead, incentivize businesses to get more serious about cybersecurity, that now we land on this issue of corporate liability or director liability.
And, and I think with AI, th- this will be fundamental. But the problem is, we won’t have decades to talk about this and think about it. We’re gonna have to build on past experience and act more quickly to work out how can this be implemented and what kind of models will work for liability. Because e- essentially, you, you could say this is [00:38:00] maybe perhaps controversial or provocative, but you could say that, you know, these AI companies are, are essentially releasing malware into the system, and, and then when something happens, they apologize some months later in, in the case of Australia.
Or I don’t even know if there was an apology, actually. I read the prime minister’s- There has, there, there has, has since been an apology … there has been an apology. Yeah. Yeah. But that, that’s not really sufficient, is it? ‘Cause that’s not how you can develop things in a global system upon which our critical infrastructure is built and runs.
That’s not appropriate. And i- if they, if they feel they need some kind of governance to prevent them from doing that, and they’re not getting it in their home country, then I guess that that’s one option for countries like Australia to consider. How can a liability framework work? And I think, you know, Australia’s an amazing example.
It’s really… Internationally, when, when we [00:39:00] have these, these discussions about tech policy, w- often looking at Australia, it just, for a relatively small country, it’s, it really punches above its weight, and in, in part due to the, the work that you guys do, but also the eSafety Commissioner. You see these things like the under-16 ban being very quickly replicated around the world.
I think about something like 40 countries now are trialing that. And I think Australia, somehow you have some kind of magic mix where you’re able to It doesn’t always work, but you’re able to put together approaches to these problems that are quite bold and innovative. And yeah, I guess that’s what would be nice to see in this context as well.
To a large extent, it is a middle power problem, isn’t it, Johanna? Because for most countries, the US and China maybe being the exception, we’re gonna have foreign technology in our critical infrastructure in [00:40:00] everything. We are gonna have foreign-owned and operated technology deep in our infrastructure, and the idea that we will create our own or have domestic alternatives in bits and pieces maybe, but fundamentally, no, that will not happen.
And so the, the great question of the next chapter, I guess, is exactly what you’re doing. How can you then govern foreign technology in a way that it allows you to extract the maximum benefits and mitigate against the worst risks? And that is a governance problem. There might be some technological solutions as well, but in large part, that is a governance problem.
But I, I think for Australia in particular, and the UK is in the same, same position, we had this long kind of easy ride where the best technology for the last many decades has been coming from our very close strategic partner. [00:41:00] Now, we need to be planning for the future where that is not happening. The best technology in, in every domain is not coming from the United States anymore.
A lot of the best technology is coming from China, and it will continue to do so, and we want that technology, and we don’t have a close strategic relationship with China. So how can we do that? We just… W- without just excluding ourselves from the future, how can we build those governance structures so that we can get what we need?
Johanna Weaver: You know, I was in a meeting yesterday actually where someone, um, was talking about, you know, how powerful the Chinese open source, uh, open weight models are, and, um, that we need to take a similar approach to Australia took with 5G and just exclude all of those models. And, you know, that is one potential option, but I think it is a small, uh, subset of the options that we have available to us.
But we have to be able to provide assurance that we actually are [00:42:00] able to control that technology and also that it’s not having a negative security implication. And the irony is, and, and, you know, so much of this conversation is actually uncomfortable, right? But at the moment, the Chinese are actually much better at controlling their model, controlling their models because they’ve got all of these regulations.
Madeleine, w- I wanna ask one last question of you before we let you go. What do you think is the biggest misperception about Chinese approaches to technology, or something that you find yourself constantly correcting or itching to, to set the record straight on?
Madeline Carr: Perhaps that everything here in the tech sector is top-down managed.
It’s one of the most interesting misconceptions that I’ve come to understand. Certainly, there is a, a kind of overall [00:43:00] centrally set framework for where the country is going and w- what the objectives are, but the extent to which those are developed locally at a provincial level or a municipal level is really extraordinary, and the competition, it’s an incredibly competitive ecosystem here.
So the, the people here are very entrepreneurial, but it’s incredibly competitive. So… And the Chinese government has no problem allowing competition to fall by the wayside, and the survival of, of the strongest, basically. I mean, there’s a, you know, there’s a story about how they invited Tesla into China, that, that, that there were many, many EV companies at one point, and they really wanted to kind of cull them, and they invited Tesla in as, as the world’s best EV company, and that, that eliminated a lot of the Chinese competition [00:44:00] because they couldn’t measure up to Tesla.
And so the… there’s a very competitive tech landscape here. They… There’s plenty of government subsidies and, um, provincial level support to attract tech companies, that’s for sure, but it’s not the case that these companies are just propped up and they don’t have to, they don’t have to succeed. They’re very competitive, and the ones that succeed are, are excellent.
And that’s why these kind of comments like, “Let’s just not have Chinese technology,” I think are so shortsighted for Australia because the world is changing, and Australia does not wanna be left on the wrong side of what’s happening now. It doesn’t wanna shut itself out because of some ideological attachment to, to America
Johanna Weaver: Thank you so much for taking the time to have this conversation.
Madeline Carr: Not at all. It’s just a real [00:45:00] pleasure to talk to you, Johanna.
Johanna Weaver: Thank you.
Well, that’s it for this episode of Tech Mirror, which is brought to you by the Tech Policy Design Institute. We’re based here in Canberra on the lands of the Ngunnawal Ngambri people. If you found today’s conversation useful or thought-provoking, please do share it with a friend or a colleague, or leave a review and subscribe wherever you get your podcasts.
If you’re watching, please don’t forget to like and follow. For show notes, you can visit techpolicy.au/podcast. And this podcast was made possible with thanks to the generous contributions from government, industry, and philanthropy to the Tech Policy Design Fund, the full details of which are available on our website.
The team at Audiocraft produced this pod on the lands of the Gadigal people of the Eora Nation. Music is by Thalia Scopellitis. A big [00:46:00] thank you also to the team at the Tech Policy Design Institute, without whom this pod would not be possible. Thank you for joining us, and as always, get in touch and get involved