Many believe that Artificial Intelligence will soon be embedded in nearly every aspect of the modern world. But how diverse are the people shaping this technology? In this episode of Tech Mirror, Johanna Weaver is joined by Dr. Gemma Killen and Professor Rashina Hoda to explore how a lack of diversity in AI development is amplifying harms towards women as well as marginalised and vulnerable groups. From biased recruitment, renter prejudices and healthcare systems to deepfakes, they acknowledge the existing challenges with AI’s track record on diversity and suggest some practical steps to improve inclusion and instead use AI to reduce bias and, just maybe, even smash the patriarchy!
Links:
AI has a woman problem. It could cost us dearly | The Financial Review
What Does a Software Engineer Look Like? Exploring Societal Stereotypes in LLMs | Monash University
ARTIFICIAL INTELLIGENCE and Women’s Health | Working with Women Alliance
ARTIFICIAL INTELLIGENCE, Gender and Equality Economics| Working with Women Alliance
Transcript
Johanna Weaver: [00:00:00] The Tech Policy Design Institute acknowledges and pays our respects to all First Nations people. We recognize and celebrate that among many things, indigenous people were Australia’s first tech innovators.
Gemma Killen: I’ve had this response of, “Yes, well, those biases already exist in the world, so the AI system isn’t making anything worse, but it will make some things better for other people. So that’s an acceptable risk to take.”
Rashina Hoda: It’s often about just looking around you to see who’s in the room and who is on the table, right?
It’s often as simple as that, and it’s usually the people who are not in those rooms and not on those tables, not having those conversations are the people that are at the most risk of being left out.
Gemma Killen: This is a great opportunity when we’re having a technological revolution to embed more equality into the systems that are gonna become so fundamental to our lives.[00:01:00]
Johanna Weaver: And I think it’s also inspiring women to recognize that this is actually an opportunity to smash the patriarchy if we get this right. I, I really believe that, right?
Hello, and welcome to another episode of TechMirror. I’m Johanna Weaver, your host, and this is the podcast where we talk about how technology is shaping our world and how we, the humans, or in the case of this podcast, women, can help shape technology back. And particularly today, we’re gonna focus in on a field of artificial intelligence called AI safety.
And actually, if you ask people what AI safety means, you’ll probably get as many different answers as many people as you ask. So for [00:02:00] some people, AI safety is looking at things like existential risk, so AI becomes so powerful that we as humans are unable to control it anymore. For other people, when they talk about AI safety, they’re thinking about more immediate harms, things like bias and discrimination in the outputs of artificial intelligence.
And what we’re gonna focus on today is what it means to be answering this question of what harms, and importantly, whose harms are important, whether having more diverse voices at the table when we’re building, deploying, and using artificial intelligence, whether that would fundamentally change the outcomes that we get from artificial intelligence.
And we have two incredible guests with us today to explore this topic. My first guest is Dr. Gemma Killen, and Gemma is the executive director of the Working With Women [00:03:00] Alliance. Her work, um, self-evidently, focuses on gender equality, but she’s done some recent research that is particularly looking at how AI can affect women’s economic participation, health outcomes, and how do we get an equitable distribution of the benefits of AI.
So Gemma, welcome to the pod.
Gemma Killen: Thanks for having me.
Johanna Weaver: And our second guest today is Professor Rashina Hoda. She’s a software engineering researcher and the director of the HumanAIse Lab at Monash University. Her work focuses on the human AI collaboration, with a particular focus on AI bias and the sociotechnical factors that shape the development of safe and trustworthy AI.
Rashina, it’s wonderful to have you on the pod today.
Rashina Hoda: Thank you for having me, Johanna.
Johanna Weaver: So when we say the words AI safety, it really does mean different things to different people. [00:04:00] So for the two of you, when you hear AI safety, what’s the first risk that you think of? And maybe it’s a risk that is under-recognized and perhaps something that others aren’t as focused on.
So Gemma, why don’t we go to you first, and then, um, Rashida, we’ll come to you next.
Gemma Killen: Thanks for the question, Johanna. I think that one of the things that we tend to overlook when we’re talking about AI safety is the harms that women face through the use, use of AI, and there’s a big spectrum of those harms.
There’s the sort of bias that’s embedded in the systems that we use now to assess bank loans, to assess rental applications, to assess job applications, that bake in the sort of discrimination that we’ve seen for decades against women. But there’s also the nefarious use of AI that happens and it targets women.
So in One s- in one example, there’s, I could buy a chatbot for my [00:05:00] website, program it to spit out disinformation about how harmful contraception is or how abortions are morally wrong, uh, and no one would hold me accountable for that. I would be allowed to just, uh, let people assume that that information is correct through the use of the chatbot.
But also, we know that there’s increasing use of AI tools by perpetrators of domestic and family violence in the family court, for example, to put forward vexatious claims, delay family court proceedings, keep people wrapped in a cycle of domestic and family violence. So those are the sorts of harms that I think about when I think about AI safety.
Johanna Weaver: And Rashida, does that echo with you? Um, and do you think about additional harms?
Rashina Hoda: No, absolutely. I think what Gemma said really captures well some of the issues we’re facing as women in this age of AI. Um, I’d also like to add from the perspective of developers and designers of AI that we know, [00:06:00] um, from recent reports that anywhere from 20 to 30% of developers, designers, and researchers in AI are women.
So that’s a very short, small percentage of the total- Mm … population. And what that means is the risk there is when we develop AI, that we might get these blind spots into our products and services, uh, the examples, uh s- uh, that Gemma, uh, shared, for instance. And that can happen because there is not enough diversity in the team that is designing these AI systems to begin with.
And then there’s the upstream effect in terms of the foundational and frontier models, the AI models, the large language models. They’re based on these massive amounts of data that is already biased against women and other marginalized groups to begin with. So you have this whole cascading effect from, right from the foundational models into the products and services because there’s not enough diversity in the teams, and then onto the users.
So it’s a chain, [00:07:00] and it’s a cascading effect, as I said. And the other aspect of it in terms of the risk, which is now being pointed out by various people, is this idea of AI competency penalty Which is to say that when women use AI or try to upskill themselves using AI at work, they, it is seen as proof of their incompetence.
“Oh, she’s using AI because of course she couldn’t have done it without AI.” Whereas if the same thing is being done by men, it’s seen as pragmatic use of new technology, they’re on with it, they’re really, you know, uh, moving with the times. So it’s a very ironic situation, and it is the, uh, as I said, the AI competency penalty that women face exclusively.
So I mean, the, the way I put it is, like, damned if she, if she does, damned if she doesn’t.
Johanna Weaver: And I wanna draw on a particular element that you’re talking about there, Rashina, in terms of the people who are designing these [00:08:00] technologies, and in particular looking at… I, I love the way you’re talking there about the foundational models.
And for those of our listeners who maybe don’t live and breathe AI, the, the foundational models are called foundational or frontier models because they actually are then used and built into so many other types of artificial intelligence. So people take these foundation models, and then they tweak them and modify them.
And so if you have bias in those foundational models, which frankly there is, because a lot of these models have been trained by learning off the internet, and we all know, um, how glorious many parts of the internet are , then you are starting to see that reflected in the outcomes that these models are producing.
We’ve spent a lot of time talking about bias, or I have over the last few years, and listening to you, it does make me wonder if we’re actually asking the wrong question, rather than focusing on the output, which is bias, whether or not we should be looking more directly at the input, which is, how do [00:09:00] we actually get more diversity in terms of the people that are making this technology?
And that’s more diversity of women, but also more diversity, full stop, of all types. So do you think that would make a big difference, and how do we do that? What are the practical steps that we would need to take to get more diversity in the people that are, that are building and making this technology?
Rashina Hoda: So that, that’s a great question, and I think the diversity problem is very real. So one of the pieces of research we did last year with our collaborators at CSIRO, uh, was to look at these foundational models, as you said, the GPT and, uh, Copilot at that time.
We made a little scenario for them to rank candidates in order of preference for a particular job advertisement. Right? And so this was all kind of, you know, an experimental setup. Uh, we made these profiles exactly same except for the gender, right? And, uh, you know, the details are available in the paper, but I’ll cut to the chase [00:10:00] in terms of what we found.
It was s- you know, not surprising in some ways that ChatGPT was hugely biased towards men and men from Caucasian backgrounds, Global North settings, and also interestingly, um, athletic build. So there, there was, there was, and there was a part of about it in the recruitment, but we also asked it to draw software engineers where, you know, you, there’s lesser chance of actually hiding the biases.
So in those drawings it was, you know, predominantly these Caucasian men who are super athletic kind of situation. There was not a single, uh, representation of anyone with a visible disability. And Copilot was kind of funny because it was funny and biased in a funny way in that it would always just pick the first candidate in the list.
It didn’t care Who the other ones who the other ones were
Johanna Weaver: Maybe it just had a tough day and it just didn’t wanna spend the [00:11:00] time doing it. I mean, I’ve been in recruitment processes where I’ve felt like doing that.
Rashina Hoda: It’s hilarious because it… Basically what it means is in a real-world context, imagine the first person who walks in for the interview gets the job.
So it… You know, and, and it’s funny because when we put these things out there, and it’s this sense of, “Oh, it’s coming from a computer, therefore it must be objective and right.” And it’s not, and it’s super biased. Coming back to your question, I think the fact that some areas are particularly challenging and not right for putting AI into, for example, recruitment.
So… And that, that is something that is looked at as being, you know, a silver bullet. “All right, we can fix this issue and just put AI into the filtering.” Not a good idea. Not with all of these biases. So that’s one issue. You have to do a little bit of extra effort if you want diverse teams. And the short list, to have a short list that is already diverse, and not just your one token woman on the list.
Uh, we- Mm … we know that doesn’t work. And then right up to the stage where they’re… The shorter list for the interviews [00:12:00] and so on has to keep getting, uh, representative. The panels need to be diverse, so the hiring panels themselves need to be diverse. And of course, there’s a whole pipeline issue, right, from how we educate our kids and, um, how we talk about AI in terms of, uh, you know, or technology generally as being something for all genders, not just for boys.
Johanna Weaver: Gemma, Rashina’s just referred there to some examples of AI being used in employment. You have some really good examples from the work you’ve done looking at in areas of healthcare and some of the biases that are emerging with artificial intelligence in healthcare. Could you talk to a, talk us through some of the findings that you came out with?
Gemma Killen: Uh, so the research that we looked at around women and AI in healthcare showed that women are more likely to use AI to fill in healthcare gaps. And we, I think, you know, we all know what happens with medical misogyny. Basically, women’s health isn’t taken seriously in the [00:13:00] real world- Mm … AI aside. But it means that women’s health can be really expensive, take a really long time to access proper care.
And to fill the gap- Women are turning to general AI tools in order to get the care that they need, get the kind of diagnoses that they want. But there’s high rates of misinformation and disinformation in, out there and being scraped by AI tools. There was a piece of research that I read recently that said that people who are peddling peptides at the moment, for example, are filling Reddit and chat boards with information that is specifically written to appeal to AI scraping tools, so that when you ask a chatbot a question about peptides, the information that the peptide companies have put out there is scraped up and delivered to you as though it is a neutral summary.
And the impacts on women are, are much more significant than they are for men because our healthcare is already [00:14:00] so deprioritized.
Johanna Weaver: And do you think that if we’re talking about diversity, is it about improving the fairness in those outcomes, or is it more fundamentally different? Does it change what we’re looking at when we’re thinking about risks?
So Gemma, why don’t I come to you on that one?
Gemma Killen: Yeah, I think which things you consider to be risks and where you look for problems depend on your life experience. And if we have only a certain group of white men developing these tools, then of course they won’t think to check whether their tools encourage misinformation spread or are biased against women in hiring processes.
And in the same way that when men are on hiring panels, they might think it’s perfec- perfectly reasonable to put the CVs that have big time gaps in them to the bottom of the pile without considering the impacts of the care load on women. When they’re developing systems, [00:15:00] they might- approach it in the same way.
So I think it’s really important to embed diversity in the development of tools from the very beginning.
Johanna Weaver: And Rashida, when we were on the panel, you had a great analogy about a toaster, which I think actually helps to emphasize this point that Gemma is making. Can you talk us through that?
Rashina Hoda: Sure, the toaster analogy.
So this was when I was thinking about your questions, Johanna, about what comes to mind when you think of AI safety, right? And this is something most of us do in the morning, is toast a piece of bread or so. And I was just thinking about the number of regulations and standards a humble toaster has to actually go through and pass before it is allowed to be inside of our homes is quite a few, right?
So it has to be checked for electrical safety, for thermal safety. In fact, and apparently there’s also something to do with a tipping point where it’s not accidentally tipped over by pets and [00:16:00] by kids. So there’s an angle at which… And then there’s the auto electric shutoff in case there’s something going wrong.
And of course, then the, with, things go on from there around fire hazards and whatnot. So this is a simple home appliance, and an amount of risk assessment around a simple home appliance and safety standards that it has to actually pass. And then the fact that the users know for a fact that this has been quality checked, and it has actually passed those standards before they are comfortable using it.
Now compare that to AI or using, you know, ChatGPT or Gemini, what have you, all of these Where are the standards? Where is the regulation? Where is the dis- yes, there’s a little disclaimer that says, “Of course, we can get it wrong, and don’t get me started about hallucination.” But seriously, I mean, compared to a humble toaster, there isn’t enough really risk assessment for something as pervasive in our lives and as AI.
And our kids are using it, [00:17:00] our adults are using it, everybody’s using it without any concern for all of the s- risks that it carries. So I just find it mind-boggling. So that’s my toaster analogy.
Johanna Weaver: And I, I think you also made the point when we spoke last about, you know, if, and if the toaster was known to shock women three times more than men.
Rashina Hoda: Thank you for reminding me. Thank you for reminding me. So that’s the other part of it, which is incr- even more shocking, which is we cannot even think of a toaster that is actually known to shock, say, young women 10 times as more as men, or to shock disabled people, or not toast, decide not to toast for people with disabilities every other Monday.
Like, we cannot even imagine such a product, and yet we see all these biases in AI where the same prompt, depending on, you know, your profile or background or whether it is a prompt about a different gender or a different [00:18:00] ethnicity and so on, is gonna give you bizarre and almost unexpected results, right?
So there’s no regulation in that sense as well. And so it’s a matter of safety, and it’s a matter of inclusion, and these are standard things that we expect from any product or service Even a whole humble toaster, so why not from something as powerful as AI?
Johanna Weaver: Yeah. And I would argue actually that Australian law already requires…
You know, we have anti-discrimination laws that prohibit this type of thing, right? The challenge is that they’re not currently being applied or enforced against the AI companies. And so how do we do that? And that requires, you know, a big shift in terms of the regulators’ capacity, but it also requires much more transparency from the companies as well.
Now, Rashida, i- in your previous response, you were talking about employment and how do we get more people to the table. I really do wanna come back to that point, but before we do, I wanna really double down again on this question of whose harms matter. So, [00:19:00] you know, you’ve both given us some really good examples there, both in healthcare and in employment.
Why do you think it is that we allow, or that society has allowed AI to roll out when we know that there are these biases, and when we know that it is having disproportionate harm on particular parts of the community? Why do you think it is that these groups are consistently having this type of disadvantage entrenched almost through artificial intelligence?
And are there other groups that maybe we haven’t yet mentioned that we think we should also be paying more attention to? So here I’m really looking at whose harms matter. Gemma, let’s go to you first.
Gemma Killen: I think one of the things that we found when we were looking at, for example, bias in rental application assessments, was that, um, AI systems routinely, uh, deprioritize applications from people who have [00:20:00] income support, for example.
And we read a few tests where people had submitted an application with their income support and without their income support, and even though the one without the income support listed had a lower income, it was valued more. So I think there is also that element of the way that people with less money, lower socioeconomic status, are also discriminated against through AI systems, and that’s not something that we routinely talk about.
And to your other point about why we’re allowing this to happen, I think it’s because there are certain discriminations or certain biases that we still consider socially acceptable. A lot of the conversations that I’ve had with policymakers, for example, about the biases in AI systems I’ve had this response of, yes, well, those biases already exist in the world, so the AI system isn’t making anything worse, but it will make some things better for other people, so [00:21:00] that’s an acceptable risk to take.
So I think there is a sense that’s just the way of things as it stands, so why would we address it now, even though this is a great opportunity when we’re having a technological revolution to embed more equality into the systems that are gonna become so fundamental to our lives.
Johanna Weaver: Yeah, and I think what you’re articulating there, Gemma, is exactly what we get to in this podcast, right?
These systems are being made by people, and perhaps those people need to be more diverse, but it’s also we get to shape these systems, and how do we, how do we have more of an active role in shaping what those systems are so that we’re reducing the inequalities? Because there is the real potential for AI to be used for good, but it won’t just happen if we don’t actually pragmatically intervene and take action.
Rashina, would there be… Are there any other groups that you are particularly focused on in terms of peoples whose harms [00:22:00] perhaps are not getting enough attention?
Rashina Hoda: Yeah, so this is something I often say, which is when we are in these conversations, say, at the AI safety forum when we were there, or when we are, you know, having these conversations at our workplace, it’s often about just looking around you to su- see who’s in the room and who is on the table, right?
It’s often as simple as that, and it’s usually the people who are not in those rooms and not on those tables, not having those conversations are the people that are at the most risk of being left out. So for example, have people with a visible or invisible disabilities. You’d have… Oftentimes, it’s actually, quite sadly, our First Nations people that are, you know, left out of these conversations.
You know, we’ll have people who are from, as Gemma said, low socioeconomic background. I’m also gonna add people from particular age groups, so kids, young adults, older people, oftentimes not considered as actively as in… you know, we should. So it’s almost like we are designing for this [00:23:00] default average human which does not exist, right?
There is no average human, and, um, all of these variations, if you like, from that average default is the reality of who we are, and we have to consider that. So it’s a, a range of people that get left behind. It’s just quite shocking and quite sad. But I was also gonna quickly add on a little bit about the people who are designing, as you hinted at, Johanna.
And in particular, there’s this acronym doing the rounds these days around MANGOES without the E. So it’s Meta, Anthropic, Nvidia, Google, and OpenAI, and SpaceX, right? And if you picture just for a moment their CEOs, it’s an extremely homogenous group of leaders, and they’re literally deciding the fate of the world.
Because this technology is just about in every product and service, and it’s gonna, you know, keep getting, uh, [00:24:00] more and more pervasive. And that’s kind of the pinnacle of this AI gender gap, as I kind of refer to it, right? So from there, then it cascades to the design groups, and the governance teams, and, uh, the developers, and then into the how people use it.
So it starts right at the top, and it cascades from there.
Johanna Weaver: Yeah. And I guess, I mean, they’re all right at the top, but also, you know, to flip it as well, right at the foundation, right? Because those tools that you’re talking about are all the tools that are being used to build all of the other systems on top of them.
And I couldn’t agree with you more about, about the, the impact that it’s having on different communities. I mean, it’s a very small and not hugely significant, but, you know, I’m chronically dyslexic, and I swear the spellcheck on a lot of products is getting worse and worse and worse because they’re investing more and more energy into making their AI models be really efficient.
And it drives me bonkers ’cause I know that these models can do this, and it, and [00:25:00] it’s really quite noticeable. Or what type of harms perhaps are not being considered, and whose perspectives maybe, um, are not present in the discussions. What I’d like to focus on now is what do we do about that? What’s the practical things that we can all focus on to change this, right?
Because this is about how do we help shape a better outcome, because we’re not powerless in this scenario. Rashina, you spoke a little bit earlier about from an employment perspective and, and getting, you know, having diversity in the panels, having consideration in the way that you’re doing the advertisement, these types of things.
Are there other practical steps that either people working in these companies can take or also listeners who maybe don’t work for tech companies but are concerned about these impacts? What can we do about this?
Rashina Hoda: As regular users, I think- The, the [00:26:00] answer is almost at our fingertips. This is not foolproof, but it’s a place to start, which is you can actually have conversations with AI about what it’s not good at.
So you can start there. And so for example, if you were looking at using AI for a specific task, like helping you with your home renovation or whatever it is, that you can also get it to critique and you can get other AIs to critique the outputs of another AI, and, and that way, where you can get them to look at, all right, what are the bits that I’m missing?
Now, as I said, it’s not foolproof, it’s a place to start. But that, that’s on an, like an everyday basis. But I actually have more hope from our next generation. So that’s where I’m… So as a researcher, I, I get to research the, um, unfortunate biases and so on, but then as an educator, I have this unique opportunity to educate the next generation.
And even though I’m not in schools, I’m in tertiary education, in schools, I think that is a huge [00:27:00] opportunity to help kids understand and young adults understand this is not rocket science, okay? If anything, one of the strongest Positives out of AI is to democratize technology. And you can learn, you can learn using AI, again, with all of the disclaimers around, you know, bias and hallucination.
But you can actually learn and start to look inside the box of this, you know, technology development. And it’s not for some complex rocket science NASA people over there. It’s all of us. So the next generation of kids, I feel like especially kids, especially girls, and especially kids from marginalized communities, they need to be empowered to own this, and look at themselves as possible leaders in the space coming up, and not just as, you know, as the people who are on the margins of these statistics that are always consistently being left behind.
So the more they’re [00:28:00] empowered to understand the technology, and really understand what’s behind this black box, that they can take charge of that in the future and actually shape it in the right direction.
Johanna Weaver: And what about for you, Gemma?
Gemma Killen: I think one of the things that always freaks me out is that we have data that says in Australia, men are more likely to have their training and their education paid for by either their employer or their government, and for women, that’s not the case.
And that creates a huge gap across a range of industries, but we know also the government has said that it’s women’s jobs that will be most displaced by AI. So there’s a real opportunity here to invest in training women in how to use and understand AI systems, particularly so that they’re not disadvantaged as the next wave of AI comes through, uh, and changes the jobs that we have in the near future.
So I think that’s one thing, is make sure we help to pay and build the structures for women to learn. Uh, and then I [00:29:00] think just always remembering to be critical about the information that you engage with, and try to think outside of yourself and your own positionality about the information that you’re being fed, uh, and how it might reflect biases that are embedded in the system.
I think asking yourself a couple of questions about what it is you’re reading and how it might have come to be is really helpful in addressing some of those critical bias.
Johanna Weaver: Gemma, if you could change one thing about the world of AI right now, what would it be to make it easier for women to be engaged, or for other disadvantaged groups to engage with AI?
What do you think the most important change that needs to happen is?
Gemma Killen: Yeah. I think it comes back to paying women to learn. Mm. Uh, we know that women’s confidence levels with AI are much lower than men’s, and, uh, women mistrust or distrust AI systems more than men do. [00:30:00] Probably rightly so. So … And, yes, rightly so.
But to make an intervention in that cycle is really important. Training women so that they can help to build those systems and make sure that they’re not causing harm to other women in the future would make a massive difference.
Johanna Weaver: Yeah. And I think it’s also inspiring women to recognize that this is actually an opportunity to smash the patriarchy if we get this right.
Mm. I really believe that, right? Um- Yeah. So, Rashina, what about for you, one practical change?
Rashina Hoda: I think we need strict laws and regulations and enforcement of those laws and regulations. You know, our prime minister talked about the Office of AI and the announcement around the national standards for AI, and made it very clear that if you’ve got, you know, infringement of the copyright of Australian artists, that it is no less than theft.
And what I feel is that we need the same level of clarity and firmness [00:31:00] regarding harm that is being perpetuated against women by AI. So this is non-negotiable. We have to have strict laws. Now, again, we have the stats. I mean, the, um, uh, the international group of, um, scientists on AI recently released this report which shows 99% of deepfakes are actually targeted against women We cannot have this.
We cannot have this. And this is a non-negotiable. So this is where the laws need to come in to actually penalize people who are doing this and who are using this. So I, I would strongly try and put this across, which is in our national standards, which are supposed to be coming up, that we include this idea that, uh, women are being left behind and actively look for legislation and laws that protect women from the harms caused by AI.
That’s extremely, yeah, serious and important.
Johanna Weaver: Yeah. And I think there’s so much, [00:32:00] Rashina, in that about actually enforcing the existing laws, and many of these things are already prohibited under Australian law, right? It’s just that we’re not enforcing them. And whether that’s through, you know, courageous people who decide to make test, take test cases, whether it’s through regulators who are enforcing it.
But I do think with the, the approach that the government is taking with the establishment of the AI office and the Prime Minister’s department, they’ve also made very clear that the existing mandates of the regulators, the existing laws apply. But the thing we haven’t seen is increased funding for regulators to actually be able to apply those laws and enforce those laws against AI companies.
And I think unless we get a few quick follows of enforcement action, which inevitably will take years to go through the courts, but to actually initiate those processes, this statement of existing law applies becomes an empty statement. And I say that as someone who was [00:33:00] on the record when the government first made that statement a number of years ago saying, “This is the most powerful thing a government can do,” say the existing laws apply because you’re not having to make new laws for artificial intelligence, but we have to enforce them.
And to me, that is absolutely crucial.
We had this panel at the AI Safety Forum. It’s been a week or a couple of weeks, um, it’ll be even more so by the time this actually drops. I attended the safety forum 18 months ago and made it my mission to ensure that there was greater diversity at this safety forum, so we had a, a Women in AI mentoring network.
Both of you were mentors, so thank you again for being mentors in that network. What are your observation, first, about the people in the room And then also your key takeout from that AI Safety Forum, which really was a key convening in terms of the researchers and experts who really are working on these issues in Australia.[00:34:00]
Rashina Hoda: So I think it was such a great idea to have that men- women me- mentoring program because… And, and to have it before the forum, right? So we met in the evening, and we had a lovely time getting to know each other, some of the women in that room. But also the mentor/mentee relationship meant that some of the more early career people could know that they have at least one other person, you know, backing them.
Um, so when we actually ended up in the AI Safety Forum the next day, it was way better experience than it would have been, I think, because I knew these faces, right? And I was like, “Oh, hey. Oh, hello.” And then you have the sense of, you know, you start to… Yeah, sure, it’s still about, say, 20, 30%. I don’t have the numbers on how many women ended up attending.
Johanna Weaver: Hmm. I’ll have to ask them, actually.
Rashina Hoda: Yeah. And, and s- it’s always funny because when you… Sometimes when you ask this, and this happens in our classrooms, my male educator counterpart would say, “Oh, Rashina, isn’t [00:35:00] this nice? There’s so many women in the class.” And I’m… I was like, wait, I was about to exactly say the opposite.
Where are the women in the class? So, so it’s, it depends on, you know, what you’re looking out for. But at the AI Forum, I felt, and I know from the fact that the year before was far less women, that we’re on the right trajectory to actually improve that diversity. Um, what I also liked in terms of diversity was people coming from a number of different disciplines, and a number of different stakeholder groups were represented.
So you had industry, you had academia, research, governance, government, policy, people who were just, you know, societal kind of not-for-profits, and so on. So it was such a range, and rightly so, because AI is touching every aspect of our lives. And so to hear what the others are bringing to the table from their disciplines was very refreshing.
And then to be [00:36:00] able to be in the same space at the same time, and we could have these conversations where we’re bringing our own piece, and we can combine these ideas, so it becomes the greater than, you know, the sum of the parts situation. So it is a really good forum to encourage the interdisciplinary approach that AI actually needs in order to solve some of its problems.
So I was super excited and very, very grateful for the invitation, and very hopeful That from the whole experience, looking at, all right, A, we’re having the right conversations, we’re talking about the latest reports and so on from the UN as well, and we are, we have what seems like the right people. The only thing I would add, I think next time we can also include the younger generation and a bit more of the older generation, because, yes, uh, we need to hear from them as well.
Johanna Weaver: Yeah. Yeah. And for anyone who’s listening to Rashina describe that forum going, “Gosh, I wish I had [00:37:00] been there,” we will put a link in the show notes because a lot of the sessions were recorded and, and you will be able to access them. Gemma, what about for you? What was your observation about the people in the room and key takeouts?
Gemma Killen: I think on the, the networking night was amazing and it was great to be able to spot people again. But the very first day when I came in, I arrived a little bit late, so as, as talk was already happening, but the first thing I saw was a woman holding a baby in the… Yeah, and I was… I’m always like, if there’s babies in the space, then it’s an inclusive and open space because it’s so easy to shut out mothers from spaces, that if someone’s made the effort to include a mother then, then there’s something good happening there.
So I think that was really promising. Uh, and I really liked, as Rashina said, like, that interdisciplinary approach and people from across the spectrum because it felt like the conversation about the [00:38:00] harms to women, the harms to other marginalized groups, was embedded in a larger conversation about the ethics of AI and what we want it to do, rather than this niche conversation which often happens where the, the women have to go off into a corner and talk about the women’s problems.
But it really felt like that that was part of the central conversation, not divided off.
Johanna Weaver: Yeah, and I was super impressed, too. We- our panel was sort of this first set of panels after the plenary discussions, and we were up against Helen Toner, who, you know, was the former board member at OpenAI, and we had a full room full of people, uh, who came and listened to our panel.
So I was particularly impressed by that. But also, we shouldn’t be impressed by that. That should be normal.
We’ve spoken a lot about the problems and the challenges, but particularly in the exchange that, that Gemma and I, uh, had a little bit earlier where we were talking about we can actually use AI to help smash the patriarchy, [00:39:00] right? This is… It doesn’t have to be this way. It can be something different. Can you help us to paint a picture of what that positive future could look like if we do, you know, roll up our sleeves and get involved in shaping AI to be different than it might currently be?
Gemma, why don’t I put you on the spot first? I
Gemma Killen: think the ideal is that we have a genuine safety by design approach to AI in which any AI system that is accessible in Australia has had the harms thought about and managed beforehand so that it’s not used to ruin people’s lives, or make them worse, or make existing harms worse, and that we have genuine accountability mechanisms, and that those accountability mechanisms Are accessible.
It’s like you say, Joanna, that we have these laws, but we can’t use them. And I think in the ideal situation, those things can be utilized to hold both people [00:40:00] that use the systems for harm accountable, and also the technology companies that produce systems that are used for harm accountable as well. Yeah.
Johanna Weaver: What about for you, Rashina? What does a world look like if we get this right?
Rashina Hoda: I can imagine a world where in the entire AI ecosystem, there is diversity throughout, right from the Mangos, which are not you know… Which are pretty much shaping, you know, the future of these models, right through to the development teams and the design teams and the governance teams and the auditing teams.
In fact, also into, and I didn’t mention this before, but the investments and the investors and the venture capitalists. And, and we know that, and again, that is a problem of only about 2% of women-only teams get funded, even though money is being poured into AI innovation. So we’re being left out there as well.
So there’s the, uh, innovation and investment part. And then of course, when it comes to women using AI, that they’re not actually paying that AI [00:41:00] competency penalty, and they’re actually being seen as progressive in their responsible use of AI, just like the men are. It’s throughout that AI, um, ecosystem that I would like to see more diversity across all the teams and all the users.
Johanna Weaver: And I might add one additional vision for us, which is that I hope that we use AI to identify bias and discrimination in our society, to help us to, to uncover places where we perhaps don’t see it happening as much, and to eliminate that. And I think that is a real possibility. This is not pie in the sky stuff.
This is within our power to demand that artificial intelligence be designed in a way that helps us to do that. I’ve loved this conversation. Thank you so much, Gemma. Thank you so much, Rashina. Um, we’ll pop, um, some links to both of your really excellent r- work and research into the show notes. And thank you so much for [00:42:00] being mentors, for being part of the AI Safety Forum, and also for giving your time again to have this conversation.
Hopefully, it’s the first of many more that we have.
Rashina Hoda: Thank you so much. Thank you.
Johanna Weaver: Well, that’s it for this episode of TechMirror, which is brought to you by the Tech Policy Design Institute. We’re based here in Canberra on the lands of the Ngunnawal Ngambri people. If you found today’s conversation useful or thought-provoking, please do share it with a friend or a colleague, or leave a review and subscribe wherever you get your podcasts.
If you’re watching, please don’t forget to like and follow. For show notes, you can visit techpolicy.au/podcast. And this podcast was made possible with thanks to the generous contributions from government, industry, and philanthropy to the Tech Policy Design Fund, the full details of which are available on our website.
[00:43:00] The team at Audiocraft produced this pod on the lands of the Gadigal people of the Eora Nation. Music is by Thalia Skopelis. A big thank you also to the team at the Tech Policy Design Institute, without whom this pod would not be possible. Thank you for joining us, and as always, get in touch and get involved