Episode 19

Harms cascade: Challenging Misogyny in AI

Misogyny and gender inequality in online and in AI systems is routine, according to two studies completed by Dr. Venessa Paech and Dr. Elise Stephenson. So how do we build safer spaces when AI can amplify harms that already exist, and why does tackling gender inequality online matter for everyone?  

In this episode of Tech Mirror, Johanna Weaver speaks with Dr Venessa Paech, co-founder and director of Australian Community Managers, and Dr Elise Stephenson, deputy director of the Global Institute for Women’s Leadership at ANU and the Victorian Women’s Trust’s inaugural feminist researcher in residence. Their respective recent reports, Misogyny in Online Social Spaces and Gender and AI: Where’s the Harm?, explore how gendered harms emerge, spread and pile up across our digital landscape.

Together, they share sobering case studies of community moderators investigating misogyny, and how AI’s gendered harms cascade through society. But they also look to the solutions already within reach, from better policy and platform design to stronger community stewardship, and consider how we can build safer, more inclusive digital spaces 

Links:

Misogyny in Online Social Spaces | Digital Rights Watch

“Harms are cascading” Victorian Women’s Trust maps 20 gendered harms as Australia’s AI regulation lags | Victorian Women’s Trust

Gender and AI: Where’s the Harm | Victorian Women’s Trust 

Digital Duty of Care | Australian Parliamentary Library

Digital Duty of Care: Make Tech Care | Tech Policy Design Institute

Select Committee on Adopting Artificial Intelligence (AI) | Parliament of Australia

Gender Compass – Plan International Australia

Books and Articles:

Space XX: Gender, Power and the New Space Race | Dr. Elise Stephenson

Transcript

Johanna Weaver: [00:00:00] The Tech Policy Design Institute acknowledges and pays our respects to all First Nations people. We recognize and celebrate that among many things, indigenous people were Australia’s first tech innovators. 

Elise Stephenson: They’re not concerned about AI safety in terms of gendered harms safety.  

 Venessa Paech: Reporting and escalating to platforms is often the fastest way to kind of get nothing happening really. It’s, it’s gonna go into the ether. There’s a lot of a strong sense of institutional betrayal.  

Elise Stephenson: We actually just have really robust policies and government systems already around gender equality. 

And so for me, I really see it, well, when that doesn’t happen, that’s a choice. That is a choice that has been deliberately made. This isn’t just women’s problem, and it’s not just having harms for women.[00:01:00]  

Johanna Weaver: Hello, and welcome to another episode of Tech Mirror. I’m Johanna Weaver, your host, and this is the podcast where we talk about how technology is shaping our world, and how we, the humans, can shape technology back. And today, we’re gonna be delving into the world of online content moderation and gendered harms of artificial intelligence. 

We’re gonna talk about what we should be doing about it, and why it matters to all Australians, not just to women. We have two excellent guests today, who have respectively produced reports on both of these topics. So we’re gonna start with Dr. Venessa Paech. She’s the co-founder and director of Australian Community Managers, and has worked in online community management and moderation for three decades. 

Who knew that we could have even been doing this for three decades? She’s also the co-author of Misogyny in Online Social Spaces, which was commissioned by Digital Rights [00:02:00] Watch. So welcome to the pod, Venessa.  

 Venessa Paech: Hi, Johanna. Thank you so much for having me. It’s a pleasure to be here.  

Johanna Weaver: We also are very lucky to have today on the pod Dr. 

Elise Stephenson. Now, she is the director, uh, sorry, the deputy director, I’ve just given you a promotion there, Elise, of the Global Institute for Women’s Leadership at ANU. She is also the inaugural Victorian Women’s Trust feminist researcher in residence, and she is the author of the report Gender and AI: Where’s the Harm? 

So Elise, welcome to Tech Mirror.  

Elise Stephenson: Thanks so much for having me, and thanks for the promotion. I appreciate that too. No.  

Johanna Weaver: No problems. Promotions for Christmas, why not? For everyone. So guys, maybe if we just start with Venessa, with you. Y- we spoke there around you’ve been doing content moderation online for three decades. 

What, how did you get into that as a thing? And I guess more importantly, how have you seen online [00:03:00] misogyny change over time in that three-year period?  

 Venessa Paech: Yeah. Look, it’s been a really interesting journey. Uh, as you know, when you do a thing for a while, you see what changes and also what stays the same, and those are really interesting patterns to pull apart and pause and think about. 

So I, I came to it, uh, like most people when they first discover the internet, you know, as a user of the thing. So I was actually studying musical theater, totally unrelated subject at the time, over in America, and started to use the internet, take part in lots of really thriving online communities. Early ’90s, late ’80s even to a point, very different technology and platforms, but still incredible communities, incredibly social. 

Uh, so I do think we have this idea sometimes that, you know, the internet only really became social with social media as we currently know, and that’s, that’s really not true. Certainly, it took it mainstream, but it’s been around a while. Um, so I started as a user- Yeah, bulletin boards and all of  

Johanna Weaver: that beforehand. 

 Venessa Paech: E- exactly right. Yeah. Which were really complex social ecosystems where all these things played out all, all the way along. So there’s a lot of lessons to be learned from a lot of folks who have [00:04:00] stewarded those spaces over a lot of years, and that’s something I’m very passionate about. So as a, a user of those spaces, I knew… 

I started to develop opinions about, you know, what good community management was and what good moderation was because I knew it when I saw it as a user. You know, I knew what it took for me to feel safe in a space and psychologically safe and safe for others to take part and for us to get what we needed out of that and, and where, how we drew our own boundaries and developed our own social norms and culture and all those sort of things. 

And then went on to forge a career in that and a f- a few degrees, and so my doctorate looks at AI and how that has come, uh, to intersect with all these things as well. So, uh, and then started to pick up work as a paid and professional community manager, which incorporates the practice of moderation. 

Community management, the first ever professional was hired in that space in about 1985, but it’s, it’s been around, and community managers sit across tons of different industries and, and sectors, so we’re sort of not an industry, but more of a practice unto ourselves, and it’s incorporative of moderation but also distinct from, if that makes sense. 

There are also people who just [00:05:00] moderate and then people who just do other aspects of community management. But it’s really fascinating to see how, yeah, what, what core sets of practices, and there are some that are pretty abiding across those platform and technological paradigms and, and different er- eras, and then what is, what has changed things, sad to say, in many ways, for the, the not better, um – Mm. 

Mm … uh, in terms of the arrival of- Your major kind of monopolistic big social platforms, which wanna sort of flatten context and flatten everybody into particular things, centered more around an individual personal profile rather than the communal. Uh, and for a long time certainly did not really put as much thought or care into s- so- social and relational designer safety as I would say some of the earliest platforms online. 

And thankfully, some of the more recent platforms as well. We’re seeing a renaissance of some of that spirit, but with newer, shinier tech now. So it’s been a fascinating journey along the way, you know, helping build a community of practice through ACM and investigating this as an academic as well.  

Johanna Weaver: What a journey. 

And just, uh, for everyone, ACM, [00:06:00] Australian Community Managers. And we’ll unpack the evolving nature of misogyny over the episode. But Elise, I wanna bring you in now. You’re the inaugural Victorian Women’s Trust feminist researcher in residence, and you made a specific decision to focus your research work on gender and artificial intelligence. 

What was the driver behind choosing that topic for you?  

Elise Stephenson: Yeah. I mean, good question, Johanna. I’m actually a political scientist and international relations scholar by training, so it’s perhaps not the natural field for me to have, um, kind of jumped into. But really what my expertise or where my expertise lies is I’m really, really interested in gender equality and policy. 

And so I spend all of my day basically focused on institutions, rules, regulations, policy, you know, staffing, a lot of government, to try and understand how can government best support society across a whole lot of [00:07:00] domains when it comes to gender equality. Now, there’s a reason why artificial intelligence is therefore very interesting to me, and it’s one of several domains that I spend a lot of time researching. 

In the kind of theoretical background that I’m at home in, it’s a bit nerdy, but hang with me. We have this really established theory that says that new institutions that are kind of founded in more gender equal times, they should be m- like more gender equal. They should actually uphold that. So when we look at things like artificial intelligence, which has seen an enormous amount of change, new actors entering the arena, a huge uptake in usage across the world, it’s one of these domains that we could classify as really fast-paced, aspects of which are really newly emerging and where there is a flood of money and attention going. 

Now, if we were to kind of consider what that means then for gender equality, it places us at a critical juncture. [00:08:00] Either, well, we just let things happen and see how things evolve, or we take this as a really specific moment in time in which we can actually influence the future directory, trajectory. So for artificial intelligence, the theory goes, okay, well, if we intervene right now while this technology and our use across society is really rapidly evolving, we’ve got the best chance to e-ingrain equality right from the start as opposed to maybe 10, 20, 30, maybe 100 years down the track when it’s so much harder to make a difference in these sectors. 

So that’s why I’m really interested in gender equality and AI because I, I really look at long-term institutional trajectories, and I think that artificial intelligence is just so widespread across everything that we do, that it is something that I guess if we can get it right from the start, I’m pretty keen to be part of that project. 

Johanna Weaver: Yeah, and I think just to pick up on something you said right at the start of that answer, that maybe it’s not a natural flow. I actually think that’s something that I push back on a lot with everyone. We, [00:09:00] we need the diversity of expertise. There aren’t people who are experts in gender and AI. There are people who are experts in AI, and there are people who are experts in gender. 

We actually need to bring this together. And to all of the people listening who say, “I don’t fit into that category of being a technologist,” you don’t need to be having really impactful work, which both of your research really shows. So you’ve both spent a fair bit of time doing the research and producing these reports, which have come out in the last month or so. 

What, for each of you, what was the most surprising thing about the research? Was there anything that kind of… You’ve both been working in and around these issues for some time. Was there anything that kind of made you stop and go, “Huh, I wasn’t expecting that”? Venessa, why don’t we go with you first?  

 Venessa Paech: Yeah, look, it’s a good question. 

If I’m being really honest, there wasn’t a lot that surprised me. I wish there was more that surprised me. Certainly, there were a few things. Um, you know, so we, uh, conducted interviews and ethnographies or, and, and basically [00:10:00] sat up close with the people who govern our digital social spaces, be they volunteers or professionals. 

So you know, your Facebook groups, your Discord servers. All over the inter- internet, there’s usually someone behind the wheel there, you know, not the platform itself, but a human there making choices about governance. What do we allow? What do we not allow? How do I deal with these issues? One thing we found, which I think would be a surprise to many others, would be just how good at this work these people are. 

Elise Stephenson: Mm. So  

 Venessa Paech: certainly it’s, it’s, for those who do it professionally, that’s an expectation we would have, that they would have a lot of skills. But regardless of professional experience really, the immersion in these spaces, the deep kind of knowledge of, of the relationships of the community context led people to be really good at something we’ve dubbed sort of contextual forensics. 

So really sophisticated work in s- pattern recognition, doing cross checks across centers of harm, extremist talking points, misogynistic voices and perspectives. Looking for them in the form of microaggressions. Looking really carefully at, you know, new [00:11:00] actors in their spaces, uh, watching their behavior, watching the behavior of existing actors shift and change, and being pulled in different directions. 

And just being really good at spotting when things feel a little off, and then developing a really strong range of responses to that. So not your classic kind of ban, block, delete, report. All of which are valid in their own way, but we do know unfortunately from a lot of experience over the years that reporting and escalating to platforms is often the fastest way to kind of get nothing happening really. 

It, it’s gonna go into the ether. Uh, and there’s a lot of a strong sense of institutional betrayal and kind of moral betrayal around that certainly with a lot of these practitioners. So they do the work themselves, and they do it in a really sophisticated way. They’ll, uh, use restorative justice techniques. 

They’ll kind of get in there and, you know, they’ll pre-bunk, they’ll debunk, and sometimes they’ll draw really firm lines and take harsher and firmer action. Not that all those work for all scenarios, of course. But I guess the key thing here- Mm … is driven by context, driven by understanding that situation. 

Also how good they are [00:12:00] at spotting it. So 86% of folks that we, uh, we reported on said they see misogyny at least occasionally, if not constantly. There’s a pretty high threshold to see it all the time. And that quite a lot of that is pretty coded and sophisticated stuff, so not your average lowball type of misogyny. 

Of course, that’s there as well. The other thing that they reported was how embedded it is, and this will be no surprise to Elise I’m sure, increasingly embedded with, you know, whatever the latest round of technology is. So of course, AI, synthetic content, deep fake content, AI memes, whatever it might be. 

Even the use of AI generated talking points from their favorite, their favorite forum or manosphere influencer. It’s being deployed a lot as a, as another tool in their arsenal. Mm. Mm. So all of these things very real and very alive.  

Johanna Weaver: And for you, Elise, so your, your report found 20 different harms related to, uh, resulting from artificial intelligence. 

Was there anything in the report, uh, or in the research that surprised you?  

Elise Stephenson: Probably a bit [00:13:00] similar actually to Venessa in terms of, you know- Mm … if you spend a lot of time studying gender equality or inequality, there might not be that much that surprises you. But in a similar vein, I think what might surprise other people is just how interlinked harms are, and I talk about it in the report as harms cascade. 

So they trickle from one area to another. And just to give you a few examples, there’s well-documented, uh, bias in LLMs and in kind of AI chatbot systems that we know artificial intelligence has learnt off kind of our, our existing knowledge base. If you just take a look at medical misogyny, for instance, so this kind of reality where a lot of medical research and health research to date has been done really specifically quite often on kind of men’s bodies as the universal body, and then women’s maybe as the, the variation to that, if studied at all in some cases. 

Also not very representative of kind of racial and ethnic diversity. Downstream effects of that, so in Australia we know that [00:14:00] whilst there are kind of similar rates of going to an AI chatbot to ask for medical help, women tend to ask about more complex health cases and conditions Which then opens us up to a whole host of risks. 

You know, is she getting the right medical advice? If that chatbot is already kind of drawing on a biased data set and research to start with, what does that actually mean in terms of flow-on effects? So we have examples like that. There’s so many more. I mean, the biased LLMs and the lack of representation in the kind of development, um, side. 

So about 30% of AI developers in Australia are women. It’s a similar proportion globally. Of course, if you look at more senior levels of leadership, that gets even smaller. But, you know, again, flow-on effects. Um, let’s just say, I mean, we’ve got a Victorian election coming up. If you are maybe asking your AI something about like, “Who should I vote for?” 

or, “What policies should I be really concerned about?” Again, are you necessarily gonna get a representative or considered feedback come back from that, [00:15:00] that actually, I don’t know, damages your interests or is in align with your interests? You know, being able to tell that is also another thing, and obviously that has downstream effects on kind of democracy and all of the rest. 

And so there’s just so many aspects and I think, you know, what I, what I find about this is some, some of the things you learn about the way in which gender and AI are entwined is kind of fun and funny and some of it’s really frightening. And I think you can look at deepfakes on the one hand and biased LLMs and a lack of representation, and you can see those things as individual kind of separately studied siloed topics. 

But more to the point, you know, what my research tries to do is connect that as a bigger picture. Looking at structural equality and structural inequality and what does that mean as a system. And I think just to throw in one more thing, which I personally I, I do find a bit surprising or a bit maybe scary, is that when we’re looking at artificial intelligence and some of these technologies that are being put forwards, I think the other point I really wanna make around this is that we are seeing a [00:16:00] concentration of power and wealth in this sector in terms of who owns this tech, who’s benefiting from it in a really kind of financial sense as well as maybe power sense. 

And, you know, I’ve just come off the back of publishing a new book on the space sector actually called Space XX: Gender, Power, and the New Space Race, and it looks at the same thing. And you look at some of the space leaders who are talking about colonizing Mars and having data centers, AI data centers in space and all these other things. 

They’re the same folk who are leading the charge on tech in AI here. Now, that’s not necessarily a problem, only when you get a concentration of wealth and power, where actually that is a bit of a problem in terms of what does it mean for humanity and how our interests are represented. 

Johanna Weaver: And so Elise, just to draw two threads out from what you’ve said there. When you were talking about h- why you chose this subject, you were talking about it being, you didn’t use the word greenfields, but essentially, like if we [00:17:00] can get this right at the start, then let’s use that opportunity. But you’re also acknowledging there that so much of what is being built into AI systems is actually bringing in the existing biases, the existing… 

And, and in many cases amplifying it. How do you square that circle, or is it a case of you actually just have to hold both things at the same time and try and make it better?  

Elise Stephenson: The point I would come back to is, because I’m, I, I guess I’m really interested in policy for, for mostly, we actually just have really robust policies and government systems already around gender equality. 

That should mean, theoretically, that how this sector develops should be influenced by that and should be regulated by that in really specific ways- Mm. Mm … that encourage equality, despite what the existing kind of knowledge base that AI is drawing on and, you know, training and under-representation, everything else. 

Despite that, we actually have the knowledge and the tools already to ingrain equality in [00:18:00] this domain. And so for me, I really see it, well, when that doesn’t happen, that’s a choice. That is a choice that has been deliberately made to ignore what we know about the state of the world and the inequalities and injustices that have been built in, and the tools that we’ve already developed for so many other domains already that kind of have helped us to produce, to reduce harm for minoritized people, or to protect children, or to, I don’t know, advance opportunities for everyone. 

I think that’s kind of, you do have to hold the two things, but I think more to the point, I’m really interested in, well, if we have the knowledge and the tools, and we have solutions, so why aren’t we putting that into practice? And I, I do think that’s a trickier question to, to answer.  

Johanna Weaver: Yeah. I, and I love that, that it, that if it continues to be this way when we hold that knowledge, it is a choice, and whether it’s a choice that we as individuals are making or it’s a choice of the people making the technology. 

I- and Venessa, the research that you have just finished paints a [00:19:00] pretty bleak picture, particularly around misogyny in online spaces. And in particular, I’m thinking here about where you’re saying that the report finds that misogyny online is just routine. It’s not exceptional. And this was the bit that I found maybe surprising, again, to your point about are you surprised, but disappointedly shocked maybe is a better way to frame it, that the presence of women alone is enough to trigger the misogyny. 

So could you speak to that and maybe give a couple of examples of what that might actually look like for people who aren’t themselves seeing and experiencing this?  

 Venessa Paech: Yeah. Absolutely. And I, and I do know what you mean. I think you’re right. It’s not, if, especially if you study it, not super shocking, but very disheartening. 

We do, I think, even as-  

Johanna Weaver: Disheartening. That’s the right  

 Venessa Paech: word. Yeah Yeah. Even if we work in proximity to this stuff, as the three of us do, it, you’d like to think that there’s triggers here somewhere, but it really d- in, in a surprising amount of the work that we did with these practitioners, and I think working across sector, that’s why this was [00:20:00] so valuable. 

Yes, we were looking at, for example, some activist spaces, some feminist spaces, but also, you know, Facebook pages of large superannuation funds and mental health for, and like, you, you name it. Complete diversity so that we got a real snapshot of our digital social commons here in Australia. And granted, it’s not an enormous sample size, but we are confident that it’s indicative of that. 

And yeah, look, even just, so a, a woman And I, I actually would take it a step further and say it’s the perception of a woman being present. For example, we had, though community management is a very feminized and gendered profession, which is in and of itself quite interesting, it is more than two-thirds female, and one of the only kind of technological disciplines where that’s the case. 

We have either moderators or community managers who might have a neutral or a pseudonymous avatar for safety reasons or just because that’s, you know, something their, their brand or their organization wants or requires them to do. But their users might assume they’re female, whether they, in fact, are or not, and then, and act in accordance with that. 

And we [00:21:00] had, you know, a female poster asking a question about a super account, for example. I’ll use that super example ’cause it is a relatively innocuous one. So the post that, you know, the, the organization has made is nothing to do with women or gender. There’s certainly patterns in what those red flags to a bull might be. 

But this might just be like, “Uh, yeah, we’ve made some updates to the fund. Go and read that document.” Something really routine and administrative, and a woman might say, “Oh, thanks for that. I’ll go check it out.” And underneath that we will then have, they might have a series of posts complaining about how women need to get out of the workforce, and they’re ruining everything, and the gender pay gap isn’t real, and family court is skewed, again, yeah, skewed toward women, and then a whole… 

And then out come the talking points, whether they’re, even though they’re completely unrelated to the topic at hand. Also, rushing to, uh, sexualize or humiliate was also a really kind of common tactic. So let’s say there is a female or a female avatar of a community manager or moderator. Comments on their appearance, again, completely irrelevant to anything. 

[00:22:00] So saying, “Oh, thanks for that. You’re really hot,” or, you know, “Go out with me,” kind of, you know, “What, w- can I slide into your DMs?” You name it, this sort of stuff, which the- their male colleagues or their, or the, or their perceived male colleagues would not be encountering. So yeah, literally just women being there in any way, shape, or form, either as the poster or commenter in question or the member of the community, or being perceived as the person or the organization sharing that content. 

Uh, pictures of women in a post. So again, if the host or organizer of the online community posts something, again, could be about literally anything. It could be about puppies, but there’s a woman in the photo with a puppy Comments about her appearance, here they come. Probably some manosphere talking points, this sort of stuff. 

Really disturbing. Important to point out that the incidences of that were, are far higher across all of those mainstream social media platforms than they are in those more independent and private networks. The reasons for that are comp- complex and integrated. It’s not a simple thing. We’ll just, you know, pick the different [00:23:00] platform, solves the problem. 

Certainly, there’s a strong track record of, of, uh, of those mega platforms being very disinterested in solving this problem and, and responding to it, and they don’t care when people report things. So community custodians or people that wanna gather and mobilize their people around an auger topic will obvious- often choose a platform where they have more configurability. 

They have the features and tool sets that let them design around what the culture they wanna uphold, let them moderate effectively, and where they can make those governance decisions. They don’t have to outsource them to a, a platform or a team of moderators in another country. Um, so that’s part of the reason, and those higher thresholds and barriers to entry. 

Johanna Weaver: Mm. A- and so Elise, Venessa’s just described the, the, I guess, the triggers to mis- or some of the triggers for misog- misogynistic content online, but also then some of the harms that flow. In your research, you identified 20 different types of harms. Could… Can you just take us through maybe not all 20, but [00:24:00] maybe some of the ones that people wouldn’t necessarily think of? 

Um, so some of the ones that aren’t necessarily the mainstream, or maybe mainstream isn’t the right word either, the ones that are top of mind for people.  

Elise Stephenson: Yeah, absolutely. I think one of the things that I’ve done, um, with the report is try to separate out what is maybe a higher severity or significance gendered harm from also what’s the strength of evidence because, I mean, anyone who’s in academia knows it might be two or three years till we get our research out, and that’s just too late, right? 

So what’s being reported in the media or even coming up on Reddit or other kind of forums that talk a little bit about, like, this is what I’m experiencing, this is what’s going on. So, you know, what’s that strength of evidence? And then also the third kind of element that I look at is durability or persistence of harms. 

And, and just to give an example, like, maybe if your AI chatbot, for instance, misgenders you, uses the wrong pronouns, for [00:25:00] instance, that might be… It depends on the individual just how severe that harm might be. But it’s also not likely to be super persistent if you then can actually train that chatbot, “Okay, well, don’t misgender me like that.” 

You know? And then you don’t have that problem that goes on for a long time. So on the other hand, so William MacAskill is a philosopher out of the UK, and he talks about artificial intelligence as being where we are right now, as being in this point of values lock-in, where how we kind of train artificial intelligence systems and the norms that we reinforce, that has a much greater potential to hang around for maybe, I don’t know, hundreds of years. 

We don’t know where AI tech is going, right? And so if we are foreclosing some behaviors but promoting others, and if we’ve already come from a, quite a misogynistic society or sex-sexist society or whatever, that could lock in, I guess, some of those values in a way that we actually don’t even know where that’s going. 

So that, that’s just to give a bit of context because I think it’s important to know, [00:26:00] like, how did I come across these harms and how do you even rank them and pull them together. In terms of some of the things that your listeners might not be thinking about, on, on the young people side, you know, I worked with quite a few young people and activists in this space, and what really concerned them were things like AI companions, where very prevalent amongst kids in schools, the use of AI boyfriends and girlfriends. 

But particularly, you know, what was being reported to them amongst some young people about just how easy it was to get an AI girlfriend and ef-effectively make her crawl around on the ground or do other behaviors which we might consider to be quite domineering, submissive, pushing of boundaries, not having those boundaries pushed back on. 

And what does that do for young people when they’re trying to establish this is how I should relate to other humans, and what are the flow on impacts to kind of gender-based violence more broadly and what is seen as acceptable and not? Then some other things that came up that were quite interesting, this is more from employers, [00:27:00] was, okay, well, hey, if I’ve got a, I don’t know, an automated system that I use at work that helps me with rostering, and it rosters on Sally with, I don’t know, Sam and- doesn’t realize that Sam has previously perpetrated some form of violence or harassment against Sally in the workplace. 

What do I do? How do you make sure that those automated processes also have kind of some of that workplace context? Other people have come to me and they’ve reported, you know, well, you might consider it more theoretical, but it’s absolutely happening already. Okay, if my, one of my automated systems or artificial intelligence, I don’t know, chatbots or whatever it is, what if they go rogue and they start harassing a colleague or trolling them or whatever it might be? 

Who do you report that to? Do you report that to IT or to HR? Who is responsible for… You know, if, if it’s a colleague, I mean, that’s a clear line of, you know, we, we’ve got to get HR involved. But what about IT and these systems? Like, how are we involving our [00:28:00] IT folk in gendered harms and being upskilled around some of these things? 

Now, you might think that some of these harms are more future state, more theoretical. I’m here to tell you they’re not. These are things that are already being reported. Again, I can say the, the strength of evidence does differ across some of these trends, but absolutely we are starting to see some of these things crop up. 

And so for me, it’s really about what can we knowingly forecast based on really gender inequality and misogyny more broadly in society. Because how we would actually theorize AI or any technology is that it influences the prevalence and the nature of harm, but we know harm already exists in society. So it’s not creating new harms per se, but it’s absolutely exacerbating the prevalence and nature of harms. 

Johanna Weaver: And I think that that point is just so important, that it’s, it’s amplifying, exacerbating the existing harms and existing inequalities. I was really struck as well, Venessa, but Elise, I th- I think this [00:29:00] came out in your report as well, where you were saying that the AI ethical workload, like the mental workload falling more to women, and Venessa, your comment earlier about m- two-thirds of moderators being women. 

What do you both think is… How do you best engage with the broader community, men and others, young boys even, on these issues in a way that helps them to become part of the solution? Um, Venessa, why don’t you go first and then Elise?  

 Venessa Paech: Yeah, and it’s… Look, that, and that’s what it comes down to, right, is how we solve the… 

I think I’d really echo Elise’s point earlier that in a lot of these cases, we don’t need to invent solutions. We have them. We have, uh, folks that have lived experience, have professional training, have diverse perspectives, know how to build great things. In so many cases, it’s just getting people pointing in the same direction and doing the work. 

And in many cases, of course, getting political will or technologic platform buy-in to do various things. I do think we need to stop looking [00:30:00] outward, and I definitely would say we need to stop looking toward folks like some of the larger platforms to solve all of these problems. Certainly, we need to continue to hold their feet to the flame, but I think that, that waiting for them to come to the table is, is a fool’s errand. 

I, I also just wanna echo, in answering your question, echo Elise’s point. Everything Elise was talking to in terms of the structural, the structurally disempowering, powering effects of some of these AI tools and their systemic biases, we see it playing out so much in, in our spaces as well. Because the platforms will often, and have done a lot in the last few years, you know, roll Fire their commercial content moderators, who are a different category of moderator who work for them, replace them with AI tools, which are, are often, although they’re getting better all the time, not so great at context, not really able to do all the sort of more nuanced techniques that I was speaking about before, and often come to the party after the harm’s already occurred or is, and it has begun very much to cascade even further. 

Moderators and community managers on the front end spend a, a huge and growing amount of time cleaning and mopping up damage from AI [00:31:00] systems and from automated systems. So rather than it liberating them to do more human work, as the rhetoric goes- Mm … they are now repairing and contextually re-stitching damage, saying, “Whoops, no, that was, that was the tool, not us, and that was the this and…” 

And again, having to repair trust where trust breaks down in those processes to earn back their social license to moderate, which is hard-won in many instances. So there’s some really interesting things there. These AI tools are infrastructural, to Elise’s point, and wrap around all of our digital social interactions as well, um, and are real life. 

They’re not separate from it. I think, you know, men are certainly a, a part of this. I think we’re seeing some really encouraging signs sort of in the broader social media discourse. I, I’m just fr- from a personal algorithmic point of view seem to be noticing more and more men taking a stand, speaking out, raising their voices, being really great strategic allies, more men advocating for women and for, um, for diverse perspectives in, in these rooms where these solutions are built, where policy is conferred. 

I think they have to have a leading role in that, a, a, a co-leading [00:32:00] role in that. And w- we have noticed that, and this is sort of a, a very common and old community technique, but the social modeling effect is real. So when- Mm … whether that’s a male ally or, um, a man or a boy that somebody looks up to takes that step and makes, normalizes that socially, gives them that license and permission to do the same, it does start to have a really positive network effect, more so than when the women are doing it. 

Also, we’ve been talking a lot with our male moderator and community management colleagues, of which there are plenty as well. They are super attuned to this as well, and that’s what’s so interesting ’cause they are sometimes behind that gendered mask of an avatar. So they get to, to walk in the shoes a little bit, and they’re full of really rich ideas as well about how to use an embedded practitionership and cohort like these folk to, who are sitting on the front lines as Digital first responders who do some early intervention work and support policymakers and technologists with other actors. 

I think literally just a- starting to have these conversations more actively is a huge part of it. [00:33:00] The differences that I’ve seen over that long period of time when these, a tool is built for a community or a moderator or a community manager without engaging with them versus when they’ve been involved in that co-design process is obviously gonna be really significant. 

So- Mm … bringing them into the conversation, demystifying who they are and what they do, and understanding that they, we have a bunch of human stewards sitting amidst all of this technical infrastructure as well, who can offer some rich insights.  

Johanna Weaver: If only every piece of technology was made in consultation with the people who are gonna use it. 

I think we would have a much better world, just full stop. Right.  

 Venessa Paech: We’d have fewer royal commissions, perhaps.  

Johanna Weaver: Yeah, yeah, exactly. Elise, maybe if I could get you to answer the same question, but perhaps drawing on your specialty, maybe zooming out and looking at it at a broader institutional level. Like, what are the lessons, and I’m thinking here about, you know, Venessa’s response very vividly painted the picture of actions that we can be taking at a personal level, uh, which does have institutional impact as you’re reshaping the platforms.[00:34:00]  

At an institutional level, are there things that we can learn from history that would help us to be taking on the attitude of many of the senior leadership, which does shape what the future of artificial intelligence would look like?  

Elise Stephenson: Yeah, absolutely. So, I mean, I think that there’s quite a few things to say. 

And just to start on that kind of point around the AI mental load and kind of this AI ethical mental load really being held by, well, reported to be held by women in my research. We need some prevalent studies, for sure. But it’s not just that. It’s also about industry. And I think a lot of the experts that I’ve spoken to in industry report that people are concerned about AI safety, as in AI not going rogue and, I don’t know, doing some sort of cyber attack and releasing all their data or whatever it might be. 

But they’re not concerned about AI safety in terms of gendered harms safety, and I would love to see more of that ethics side of the piece- Mm … really be part. I think part of this also comes down to this isn’t just [00:35:00] women’s problem, and it’s not just having harms for women. I mean, I talk about gendered harms because men also receive harms and non-binary and trans people also receive harms. 

For instance, a lot of young people, young men kind of report being targeted by Nudify a- ads. And what does that do if you’re constantly being targeted by ads saying, “Oh, Nudify your classmates, Nudify your coworkers, Nudify your friends”? Mm. Like, that doesn’t impact very well on their mental health and other things as well, right? 

Johanna Weaver: A- and that was quite, that was quite shocking to me, Elise, looking at the report, how much young boys are targeted with advertisements for Nudify apps. Mm. Not being a young boy, I didn’t realize that was a thing. I mean, that’s, that’s pretty crazy.  

Elise Stephenson: Yeah, absolutely. And, and even just, you know, we, we know about some of the stats. 

You know, there’s one UK study that tested kind of what were the images generated when it came to, well, imagine occupations for men and women. You know, women in 30% of cases were pictured as a model or sex worker. But what we’re finding for men [00:36:00] is that they’re less likely to be pictured in roles that are traditionally female dominated or in more caring professions or in some other sorts of professions and having s- the skillsets of, you know, human empathy and all these other sorts of things, which also impacts negatively on them. 

So I think it’s really important, just going back to that point about how do we get buy-in from across society. I think that the reality is it can be really easy to grasp on the gendered harms that are impacting women because they are really quite significant and quite great. But- Gender inequality harms everyone. 

You know, in terms of kind of broadening the circle and making sure we get, get buy-in. Mm. I think that’s really important to the conversation. What we have learned from other domains of work, organizational studies on gender equality and all of the rest, we know that you’ve kinda gotta be quite targeted. 

So if there’s lots of us different actors working in this space, and we’re all quite passionate about reducing gendered harms or online misogyny or whatever it might be, there’s a big laundry list of things that we need to tackle, and I think that it’s really [00:37:00] important. What we know from the research is actually some of those initiatives will be more successful if we just narrow it down to one or two priorities first, knock them over. 

The second thing is it’s really, really important if you’re gonna institute some sort of change process around to gain more gender equality, whatever it might be, timing that with broader organizational or societal support. Like, is there something else that’s going on in the world right now which means that we care about these issues? 

I mean, many years ago, Brittany Higgins in Parliament, you know, everything that occurred for her and raising this conversation on sexual harassment in parliamentary workplaces, that was really, really helpful for getting a code of conduct then into Parliament. So there’s kind of this broader support because of the egregious things that had happened, you know, to many women that, that helped lay the foundations So if that’s two points to be targeted, look for what are the, the broader supports out there. 

The third thing that is shown to be quite successful in, in producing change initiatives is, well, it [00:38:00] sounds a little bad, coercion or threat of loss. But what I mean by this is, hey, maybe it becomes part of organizational, I don’t know, chief operating officer or your chief of technology or, or I don’t know, if you’re a platform, maybe it’s one of your KPIs is to make sure that your tech is safe by design and that it has considered what are the gendered impacts, and it has gone ahead and it’s taken some actions and some steps to mitigate, reduce, or remove those harms. 

If that was part of your KPI, we know from broader research if you tie gender equality indicators to KPIs, and let’s just say your chief doesn’t get a bonus if he doesn’t do that thing, that’s actually quite effective. So, you know, that’s just a little example. Um, you know- Mm. Mm … obviously regulation is another threat of loss coercion thing. 

You know, you might lose your license if you don’t do this correctly, blah, blah, blah. So we’ve got some systems level tools that I think are really important. I think that really also leans into what we’re seeing with the Digital Duty of Care kind of proposals right now, that [00:39:00] we, government knows it needs to act on this. 

What I would love to see is systems, platforms, developers coming to the table in a really proactive way because it’s, we’re really talking not just about individual harms, we’re talking about system and societal level damages to democracy and all of these other things that should matter to us here as Australians. 

Johanna Weaver: Mm. Yeah, and so this podcast will be being released the week that we expect, we hope the Digital Duty of Care will pass through Parliament, and one of the, basically it’s a duty to provide safe online spaces, and preventing and restricting gender-based harm is one of the categories of online harm that is defined in that piece of legislation. 

So we’ll be watching that very, very closely. I think I love those, that articulation of the three ways, um, Elise. I think that’s really practical for us to keep our minds focused on, and I am gonna come to each of you and ask you what your priority action would be. So there’s your forewarning to think [00:40:00] about that. 

But I also think it’s important to pick up on this point that you made, Elise, about the broader what’s at stake and the fact that inequality is bad for everybody. So maybe just 30 seconds from each of you on how you think about- What is your elevator pitch for why everybody needs to care about equality, why everyone needs to be addressing misogyny online for you, Venessa, and then I’ll come to you, Elise  

 Venessa Paech: There’s a lot of evidence around this, right? 

And not, not just these two reports, but, uh, but decades and decades of numbers that point to this stuff. Businesses are more profitable when they do this. Products are more diverse and more sustainable. They’re better. Um, so there’s sort of, like, practical, kind of old-fashioned, hierarchical, v- very neoliberal ways to look at this. 

But change comes from community, and I will of course say that as a community person. But the systems and the direction of travel that everything is on at the moment is, not to sound … at the risk of sounding [00:41:00] a bit doomscrolly, we are heading toward a few cliffs, I think, right? So unless we solve these big intractable problems, and to do those big intractable things, the, the climate change problems, all the challenges around a- AI as, as a, as a new technology set, and everything else, wealth inequality, all of these things have come about through unequal systems, systems where, you know, minoritized groups have been structurally disempowered or exploited or worse o- over centuries. 

We just need to get this right. We need to be better than this, and I think that we as a entire human race are seeing already and suffering so many, disproportionately suffering the effects of that. And to me, this is at the heart of it. So much of it comes down to white supremacy, comes down to misogyny, comes down to people being very comfortable working in these unequal systems and taking advantage of them, and it’s gonna lead to really awkward conversations and horrible confronting conversations that might be worse before they get better. 

But I do think that we are, we’re in a bit of trouble as a species if we don’t have these big [00:42:00] conversations. And the, and you know, to Elise’s point and core lens here, you know, it is all completely cascaded through. It is all completely interconnected. And I, the same patterns are, are there that they, that were 30 years ago when I was on the internet, but hyperscaled now, of course, um, algorithmized and all the rest of it, and now monetized and productized in a way that they weren’t 30 years ago. 

But I think, yeah, it’s not great for us if we don’t figure it out.  

Johanna Weaver: Yeah. And I think both of your reports talked about the chilling effect, right, and of this, so women withdrawing from public spaces, the impact that has on democracy, for example. So Elise, what’s your elevator pitch for why everyone needs to care about this? 

Elise Stephenson: Yeah. Look, it, I think it comes down to, at its base level- Thinking about your community, your parents, your siblings, your friends, your broader network, we should want them to have the freedom to choose how their life goes, the types of jobs they get, [00:43:00] the freedom to live their lives without violence, the freedom to live safely, the freedom to pursue opportunities. 

I think that really does matter, and I think that at the base level, I mean, absolutely there are gonna be some people who do hold exclusionary views, and that is part of what they believe. But I think for a lot of people, resistance to gender equality or doing something about it is, is not about that. I think they would agree with the premise. 

Maybe they don’t fully understand it or they don’t fully live that experience themselves. So I think some of it, this is about having a great deal of empathy for fellow human beings and kind of this collective vision for what a better society could look like. I think just for any audience members who might be thinking about how do they talk to some of their colleagues or friends or families, there is a phenomenal resource by Plan International Australia called The Gender Compass, which helps… 

You know, it’s basically been this huge amount of research that has tested what is the kinds of language that resonates with the most Australians, and [00:44:00] whilst most Australians might not actually really resonate with the terms of, like, gender equality, they do resonate with terms like having the freedom to choose their future. 

And so I use a lot of terminology around free and freedom, and maybe that marks me out as being a bit US-centric at times, but it’s not about that actually. The- we know that these words actually resonate with Australians, and the values resonate with Australians. So that would kind of be my pitch for why this should matter. 

Johanna Weaver: Thank you, and we’ll, we’ll put a link to The Gender Compass in the pod notes, and I think finding those, those language sets is so important, and it is, the freedom resonates with me in the sense that it’s what distinguishes us from many other societies that we don’t want to, and unfortunately, I would put the US increasingly in that category, right? 

Which is, again, quite confronting to be thinking about in 2026. 

So let’s move away from the confronting. Let’s move to the empowering. I’m [00:45:00] empowering both of you to implement, with no budget constraints, one major change that you think, “If we could just make this one thing happen, it would have a really big, positive impact on the world.” Venessa, and I guess specifically in terms of gender or, or misogyny online, let’s not go for world peace, but, so 

Venessa, what would, what would yours be?  

 Venessa Paech: I’d say that’s huge. That’s amazing. What a great question. Um, look, I think I’ll keep it, you know, germane to my world, right? Which is that healthy commons, healthy communities, health- healthy societies do not happen by accident. People grow them, and people steward them. 

And yes, we have the helpful tools and all, all of the things that we all deal with along the way. They’re all part of the, the solution, a- as they are the problem, but we have to invest in people. So obviously, for my concerns and my areas of research, that is the frontline stewards of people who are gathering around, around circumstance happenstance or a million other [00:46:00] reasons, purpose, interest, uh, goal, common mission. 

That’s always gonna be who we are as humans. So the more we- Mm … can invest in, train, and empower community stewards to do that work ethically, um, reflexively, with, with great tool sets and in, in a way that kind of honors their contextual agency, gives them the freedoms that Elise was just talking to, I think that’d be really game-changing because we… 

And we have evidence to support that, that when those things are in place in small scales, they can have a huge effect. We’re not gonna… It’s not… We’re not talking about one big grand community, world peace. We’re talking about a networked ecosystem of lots of this- Mm … uh, and giving those people the freedom and the, the human investment and the runway to do that work. 

Johanna Weaver: I, yep, you’ve got my vote. Well done-  

 Venessa Paech: Yay …  

Johanna Weaver: tick, tick. And Elise, what about for you?  

Elise Stephenson: Yeah. Look, if you’ve only given me one thing, Johanna, I think it would be a redistribution of tech ownership so that the users are- I mean, can I second that one  

 Venessa Paech: too?  

Elise Stephenson: Yeah. The users and the [00:47:00] beneficiaries- 100% … are the owners, and the workers are the owners, and that we have this truly representative ownership model, this truly representative development model, and this truly kind of representative kinda use model so that we’ve kind of got that connection all the way up the chain so that those who benefit create, those who are harmed influence the design to mitigate harm in the future. 

I think that’s a really closed loop system, and I still bang on about, and, you know, I’m really dedicated and disappointed to see, you know, we’re still in a reality where, just as an example, women-owned business only gets 1% of total global procurement. And in the last year, women-owned, female, all-female tech startups in Australia only got 2% of all funding. 

I mean, this is egregious. We really need- Mm … to be changing some of these stats, and I think that that would do a whole lot of good in terms of flow-on effects, too.  

Johanna Weaver: Yeah. I think there’s a lot to be said for getting more female leaders, uh, into some of these positions, and then the flow-on impacts [00:48:00] that will happen from that. 

Yes. I’m  

 Venessa Paech: with you 100% on that It’s, it’s the halo cascade effect, rather than being negative and half or one.  

Johanna Weaver: Mm. Indeed. Indeed. Well, look, um, thank you to both of you for the incredible research and work that you do. If you have one piece of advice for people who’ve been listening to this conversation and maybe are feeling a bit disheartened rather than excited, like what is it that keeps you positive in light of the research that you do and keeps you focused? 

And then we’ll draw to a close on that positive note.  

Elise Stephenson: Look, I’m, I’m a real active doer. There’s so many mechanisms at your fingertips where you can have a say. So if you don’t like that particular AI model, buy one that you think is more ethical and do a bit of research. If you don’t know much about, you know, AI or tech, uh, I don’t know, find great podcasts like, um, well, this one. 

In the Blink of AI. There’s lots of other ethical ones that are discussing some of these tech. But also, there’s things like parliamentary submissions. Anyone can make a submission to Parliament, whether it’s on the digital duty of care or any other sort of bill. There was a joint select committee recently on artificial [00:49:00] intelligence. 

There’ll be more. Have your say. Get your voice out there and, of course, write to your politicians. And at a really local level, have conversations with your family and friends and others who maybe aren’t on the same page, too.  

Johanna Weaver: Yeah. Thank you. That’s all excellent- Yeah … advice. And what about for you, Venessa? 

 Venessa Paech: Hard co-sign on everything Elise says. I think I would o- only add, go down some joyful internet rabbit holes. They do still exist. It’s hard to believe- … for all the reasons we’ve been talking about. And, uh, but there really are. You know, I, I was making allusions before to some of these smaller independent platforms where we do see things like Elise was talking about. 

We see people who are taking the means of production back. They’re owning. You have economic agency. They’re building their own tools. For example, I’m, I’m quite good friends with the amazing team over at Black Sky in the US who have taken come, some of the protocols around the, that decentralized, uh, platform and have sort of built their own community centric tool sets that are just incredible, and they’re giving rise to and empowering amazing communities and creators. 

There are people doing such impressive work. They are already building that future. It’s [00:50:00] just over there, and our media doesn’t really give it a lot of oxygen, and most people don’t know where to find it. You know, if you don’t know where to find it, I’m sure you and all of us will point folk in the right direction. 

But you can look for these things. Ask folks, “What’s the most inspiring, amazing community you’ve found online? Can I join? Where, where can I go and learn about them?” Just to remind yourself that these spaces can give rise to some beautiful, incredible stuff.  

Johanna Weaver: Yeah. Yeah. I think that’s excellent advice. So thank you so much, both Venessa and Elise, for this wonderful conversation, but also for your reports. 

We’ll put the links in the pod. And no doubt, we’ll have you back sometime soon. Thank you so much.  

Elise Stephenson: Thanks, Johanna. Thanks for having us. Thanks so much for having me. 

Johanna Weaver: Next up, we will be rounding out the year with two episodes, one looking at how technology has changed and evolved in 2026, and my gosh, when you think back to the things that have happened this year, there has been a lot. And then we will wrap up the year [00:51:00] with a revision back to the podcast that Zoe Hawkins, my co-founder, and I recorded at the start of the year, and give ourselves a bit of a score to see what we got right and what we missed in our predictions for 2026. 

That’s it for this episode of Tech Mirror, which is brought to you by the Tech Policy Design Institute. We’re based here in Canberra on the lands of the Ngunnawal Ngambri people. If you found today’s conversation useful or thought-provoking, please do share it with a friend or a colleague, or leave a review and subscribe wherever you get your podcasts. 

If you’re watching, please don’t forget to like and follow. For show notes, you can visit techpolicy.au/podcast. And this podcast was made possible with thanks to the generous contributions from government, industry, and philanthropy to the Tech Policy Design Fund, the full details of which are available on our website. 

The team at Audiocraft [00:52:00] produced this pod on the lands of the Gadigal people of the Eora Nation. 

Music is by Thalia Scopelis. A big thank you also to the team at the Tech Policy Design Institute, without whom this pod would not be possible. Thank you for joining us, and as always, get in touch and get [00:53:00] involved