In a world of AI, what does it mean to be human? In this episode of Tech Mirror, host Johanna Weaver speaks with one of Australia’s foremost ethicists, Simon Longstaff, from the Ethics Centre. Together, they make the case for ethics in the wider AI discourse. They discuss why Simon urges Australians to demand more than a social licence for AI arguing instead for a social compact. Johanna and Simon explore how concepts like purpose, trust, and accountability can help guide the responsible development of AI while confronting “just because we can do something with AI, should we?”
Links
Will AI be the risk we chose not to see? | The Ethics Centre
The Ethics Centre
Australia’s AI Ethics Principles | Department of Industry, Science, and Resources https://www.industry.gov.au/publications/australias-ai-ethics-principles?utm_source=chatgpt.com
For transcript and full show notes visit techpolicy.au/podcast
Transcript
Johanna Weaver: [00:00:00] The Tech Policy Design Institute acknowledges and pays our respects to all First Nations people. We recognize and celebrate that among many things, indigenous people were Australia’s first tech innovators
But I love that you’ve managed to link in there mortality, chips, and craft beer all in one response And why wouldn’t you? Exactly. It’s amazing
Simon Longstaff: Too often we develop technology without a clear sense of its purpose. There’s an old syllogism in ethics which asks, “What is the purpose of a knife?” The answer, “The purpose of a knife is to cut.”
What is a good knife? A good knife is a knife that cuts well, and the blade must be appropriately sharpened and shaped for the task it has to perform. It starts with that purpose. When we come to the application of this technology, and even at the earlier stage, let’s not think about deploying things of great power and [00:01:00] utility in conditions of blindness where we just don’t see what we’re doing.
It’s like walking in a fog over a kind of edge of a cliff into an ethical death trap
Johanna Weaver: Hello, and welcome to TechMirror. My name’s Johanna Weaver, and I’m your host, and this is the podcast where we talk about how technology is shaping our world, and how we, the humans, can shape technology back. And today we have a really special guest, Simon Longstaff. Simon is the executive director of The Ethics Center, and is widely recognized to be one of Australia’s leading ethicists.
Simon and I serve on an advisory board together, and he’s one of these people that I’ve never had a boring conversation with. So Simon, welcome to the podcast.
Simon Longstaff: Thanks. No pressure there, Johanna.
Johanna Weaver: No [00:02:00] pressure at all. And actually, I mean, the origin of this conversation, Simon, started, we’re on a advisory board together, uh, and we were having a conversation in one of the breaks about how much angst and anxiety there is in the general population around artificial intelligence at the moment, and we were talking about this concept of social license and the fact that it seems to be, you know, the term of the hour at the moment, that everyone’s talking about the need for companies to have social license, to make people feel comfortable in using artificial intelligence.
And you challenged me, and you said that you thought the social compact was a better phrase, and that it… We should be aiming for more than social license. So, let’s start there, and, and can you explain what the difference between those two is, and why you think one is better than the other?
Simon Longstaff: Yeah. So the evolution of these concepts does begin, as you suggest, with the notion of a social license.
So that’s an [00:03:00] intuitive sense that if you are going to be coming as a legal person, which is the modern form of the corporation, into a community, you need a license in order to be able to pursue your interests. And the concept of a license is that, in a sense, you pay a fee in order to receive a license, or you pass some kind of test, and then you’re given the liberty to go about pursuing your own ends.
Now, that’s a very transactional notion, and it’s a bit like you come into somebody’s house and you pay rent. You say, “Well, now I’ve got my room, and if I wanna make it into a stink hole of some kind, that’s my room. I’ve paid rent. I can do what I, I like in it.” And as long as you continue to honor the transaction, you sort of get on in your own separate way.
There was an evolution in that thinking that then came about by two academics in Harvard, Porter and Kramer, around the notion of shared value. That is, [00:04:00] rather than just seeing yourself as being an occupant of a space for which you paid a license, you exchange some value back to your host. And now I may be doing them a disservice in the way I’ll present it, but it has the slightly parasitic notion to it that you have a host and you’re a parasite, and it’s got some kind of symbiotic value of exchange there.
And then there’s this concept of social value, which has emerged only very recently and only in limited context, but particularly has been championed and now given very specific practical application by BHP. Which is the idea that rather than you being a parasite that lives within the host called society, you are in fact an organic part of it.
In other words, it breaks down this distinction that sometimes is made between the state, civil society, and business. Said, “Well, actually, isn’t business part of civil society?” And so [00:05:00] the analogy, the organic analogy that I would use is to say, well, imagine a human body. It has a number of organs, of which a corporation might be one.
We’ll, we’ll say it’s a, it’s a heart, for example. Now, its health is entirely dependent upon the health of the wider body of which it is a part. But likewise, so is the health of that body. So instead of this being one which is based on transactions Or just a kind of parasitic exchange of value. This is an organic whole which has different elements that perform different roles.
And when you look at it in its application, it has some very profound implications. For example, the heart no longer claims any essential superior position vis-à-vis any other organ because all of them are important. And so you’ll see even a very powerful corporation like BHP voluntarily choose to subordinate its power and to sit down on the same side of the table as other parts of the [00:06:00] community, of which it is just one part of the larger whole.
And that’s quite a different model, both in terms of its qualitative aspects, but also in terms of what it then practically implies about what you do. And you can take that richer notion, as I would see it, and then apply it to a whole range of things, including the discussion about AI.
Johanna Weaver: I love that notion of looking at it as the human body and all parts needing to be functioning, otherwise, what’s the point in being alive or you can’t be alive.
If we were to take that from the abstract to looking at what that would mean in terms of building a social compact for the Australian population, what would that look like in terms of the conversations that government, the tech companies, civil society, and Australian citizens, what types of conversations would we need to be having?
And how would that be-
Simon Longstaff: What would happen, first of all, you would start to reposition the [00:07:00] whole conversation where everybody, whether you are government, which is of course a, a big gorilla in the room in, in any state like Australia, or the largest corporation or the smallest, or just a citizen. And you’d say, “Actually, rather than breaking us into fractions that therefore compete for power or opportunity, that instead we see our competition, which will always be there, as being something which is directed towards a larger common good.”
In fact, this harks back to the earliest arguments that were put forward by people like Adam Smith for the very notion of a free market, which in his view had no intrinsic value. It was simply a mechanism by which if it was instantiated, it would lead to an increase in the stock of common good. That is, everybody would be better off because we’re all in the same proverbial boat together.
So you would start off with that as an idea, and then practically you would start to see even the most [00:08:00] powerful of the tech companies actually choosing to say, “We won’t sit across the table as a antagonist or, you know, against the wishes of government or a people where we arm wrestle them to a position which we can live with.”
Rather, you would say, “Actually, we’re part of this society. Even if we are based overseas, we know that in the way we present ourselves, say in a country like Australia, we are part of the Australian story, the, the Australian political and social and economic conditions. How do we all work together in a way that ensures that we are as a whole actually not merely being sustained, but flourishing?”
Which is not to say that an economic actor wouldn’t be pursuing profit or that there wouldn’t be a regard by governments for the social welfare of its community and a whole lot of other things. But you don’t start do- down on the basis of an arm wrestle to try and see what you can extract from the relationship rather than you have that practical [00:09:00] conversation.
Now, it would be a very different set of conversations we would be having if that was the starting point than what is currently the situation.
Johanna Weaver: And how do we change the starting point? Because I think many people listening to that will say, “That sounds excellent. Yes, we should do that.” But how do we, how do we get there?
That doesn’t reflect the power dynamics in the modern world. So what would– how do you respond to that?
Simon Longstaff: It’s gonna sound like a, a rather abstract answer because I know people will say, “Oh yeah, that’s all very good, but it’s kind of pie in the sky.” But one of– I think we underestimate the power of ideas in these things and, and the power of language and the fact that you can have a concept.
If you’ve grown up in a world which assumes that various actors are In a sense competing for their own interest and, and that somehow, and this is the kind of the logic of part of the market, that it, it presumes that, for example, creatures within the marketplace are not at all concerned about the general wellbeing, that they’re just [00:10:00] pursuing their self-interest, but that ultimately that will lead to that increase in stock and in that stock of common good.
If you said, “Actually, no, we’re going to have a bit more of a nuanced view that, of the kind that actually Adam Smith does put forward in his wider writings.” And you think that’s what life could be like. No, not, not just could be, but it should be like that because that’s going to lead to a society in which we all flourish.
Then the fact that you can imagine this now, that you can actually invoke that language, that you can actually start to insist on institutional arrangements in which that become the starting point rather than treated as an exception that could never be realized, then I think that’s the beginning. Now, I know it’s not as firm as creating an institutional framework or some kind of tangible structure, but I think those things follow from ideas about what is possible.
And once you get enough people saying, “Ah, yes, I’d like to be part of that. How do I do it?” [00:11:00] Then you start to co-create the environment in which that becomes true.
Johanna Weaver: I couldn’t agree more that it is the power, the– or in the power of ideas. And what we’re seeing at the moment is very clearly an awakening of the Australian population, but I also think globally of concern about what AI is going to do, the impact that it’s going to have on their lives, on the environment, on the people they love.
And so I think there’s real power to what you’re saying right in this very moment, that this is a moment where we can shape the future. Um, but we actually do have to recognize that we have the power to shape the future because if we don’t step up and recognize that, then it’s just gonna continue on the trajectory.
Simon Longstaff: It is the greatest source of oppression of human beings is not that we are powerless. It’s that we are led to believe that we are powerless. Now, the– it’s an incredible thing to realize that the most powerful force on this planet, limited only by the laws of physics, is human [00:12:00] choice. We often think, “Oh, but that’s only the choice of the people who have formal power, institutional power.”
When in fact what we know is that when people make very tiny decisions in large numbers, that can tilt the world on its axis. So one– Look, the world celebrates heroic individuals who shape the narratives of history because of their singular actions. But that isn’t actually what drives most of the change in the world.
It’s people just falling a little bit, just a tiny bit on the right side of any question. But if you are told that you count for nothing, that your choices don’t matter, that you are powerless, and you come to believe that because when you look at the size of the challenge, they seem so overwhelming in their size and complexity, then you can lay down that opportunity to be the person who makes the choice [00:13:00] that might have been like the straw on the proverbial camel’s back.
It’s that last straw, right? For you just made that final decision that tilted the world. And so I think it’s incredibly important that we understand that with AI, that it’s not– there’s nothing inevitable about it. It, it is a product of choice, and yes, you might think that it’s the great tech bros whose choices ultimately matter.
No, they are susceptible as anyone has been through emperors, kings, presidents, and anyone else to the large scale opinion of humanity as a whole.
Johanna Weaver: And so when we’re talking about the tech bros, one of the things that they’re really, really good at talking about at the moment is the potential of AI. So can we do this with AI and the potential for it to have this incredibly transformative effect, which we are already seeing, so it’s not something- Hmm
that is just in the future. You often talk about should we, and injecting that [00:14:00] frame into the conversation. So for people listening to what you’re saying, “Okay, you, you’ve convinced me, Simon, I’ve got some power.” What’s the, what’s some framing that you can give them in terms of ways to think about whether AI is going to bring a benefit?
So not just the can it, but should we?
Simon Longstaff: So you’ve already answered one of the most important things as a principle, and that is that can does not imply ought. And the effect that you can do something doesn’t mean you should do it. And we know intuitively this is the case. There’s lots and lots of things we could do that we choose not to do.
Mm-hmm. And the fact that we spend– In most of our lives, you know, we wander around the streets of wherever we happen to live. We’re not worried about being mugged for most of the time or threatened by someone. It’s not because there’s a policeman on every corner or surveillance. It’s because we just carry within ourselves the fact that, yes, we could do all sorts of terrible things to each other, but we choose not to do so.
Not for fear of being caught, but because that would be, as we understand it, [00:15:00] simply a wrong thing to do And so that sense of self-regulation can be brought to bear in relation to the application of this extraordinary technology. Now, the second thing about– that we can do as a way of framing this is to ask when we come to the application of this technology, and even at the earlier stage at– when it’s first been developed, is what is the purpose that it is to serve?
Uh, because it’s only if you understand its purpose that you can understand whether or not it is a good expression of the realization of that purpose. So there’s an old syllogism in ethics which asks: What is the purpose of a knife? The answer: The purpose of a knife is to cut. What is a good knife? A good knife is a knife that cuts well, and entailed in that is the idea that the blade must be appropriately sharpened and shaped for the task it has to perform.
But it’s pointless having a blade which is perfect in that respect if it can’t be used because it’s attached to a handle the size of an elephant. You know, no one could pick it up. No one could use it. [00:16:00] So there’s something about fitting in the hand of the user. But it starts with that purpose. Too often, we develop technology without a clear sense of its purpose.
So, for example, in AI, there’s a massive difference between developing an AI stack, uh, which is simply about bringing about greater efficiency, where you seek to reduce the unit cost of producing whatever it happens to be. If that is your purpose, and that is your only purpose, then you invest that idea into the thing you’re making.
That’s radically different than if you have a purpose which is better to realize the mission for which your organization has been established. So that’s the second element to it, to understand purpose. The third part of a framework which any of us can inter– So I’m just thinking here. Now I’m an ordinary person saying, “Oh, what do I think about this bit of AI?”
I’d be asking, “Was it not just something that’s been done because it can be done, but can they explain these things? What do they say is its purpose? Can I [00:17:00] see that being realized?” The next stage would be: To what extent have they been explicit about the core values and principles which are informing how this has been developed and how it’s supposed to operate in the world?
Again, this isn’t an abstract question. Good software engineers should have things called markdown files, uh, when they say, “This is the framework within which the AI has been developed.” You should be able to see that, and you should be able to know that the system is transparent enough such that an expert and disinterested person could actually look at it and say, “Yeah, I can see those values.
I can see those principles being given effect through the way in which they’ve at least tried to make this work.” And there’s– We’ll come back perhaps to some of the limitations in this. The next thing you would ask is: Have they been clear about the affordances which they have created through the establishment of this technology, this bit of AI?
An affordance is A capacity to do something which is either deliberately brought into existence, for example, in pursuit of a purpose, or you can [00:18:00] create unintended affordances which create new opportunities which might be dangerous and damaging. Or you might create affordances, or you may fail to, to rule them out.
So classic example is, say, a, a handgun, uh, where they’re prevalent in a country like the United States. A person can walk in and pick up a handgun, and they can just point at it, and if it’s loaded, they could shoot anyone irrespective of who they are. Now, that’s an affordance which exists within the handgun because there’s no control over who gets to use it, whether they’re licensed or otherwise.
Whereas if you built into the grip a fingerprint recognition system so that it could only be activated and used by the person who is the licensed holder of that, immediately you shrink the possibility of it being used for purposes other than for which it was created and licensed. So these are practical things that you can do to both think about how you create those technologies and, as a person who’s curious about them, to decide do you approve or not approve, [00:19:00] what’s the framework within which you should assess them.
Johanna Weaver: One of the other things that I really love is your explanation of the difference between ethics, law, and morals, which are often used syn- like they’re, they’re synonyms, but which you really clearly articulate the difference between them. And I think that’s important when we’re talking about, particularly around governing AI and the frameworks i-in which we’re going to allow or maybe not allow this technology to be shaping our society.
So can you talk that through for us?
Simon Longstaff: Yeah. So let’s just start with law. Law, of course, is a series of constraints which are typically agreed upon, certainly in a democratic society, sometimes imposed in more authoritarian societies, that in their more usual form tell you what you cannot do. So the, um– And that, uh, certainly in liberal democracies, the law tries to allow the largest amount of freedom for a person to exercise their own judgment, live their own life, and so it will say, “But you cannot do this,” and then it will constrain.
[00:20:00] Although there are exceptions to that, of course. But it’s a rule which, uh, is typically imposed by some legitimate authority or something claiming authority, whether it’s legitimate or otherwise, and which is imposed to the extent that if you violate the rule, then it has certain penalties often attached to it, fines, imprisonment, and other forms of punishment.
But the thing about law is that law is indifferent to whether it is just or unjust. So in Australia, just as existed in South Africa, in Victoria in the 19th century, there were anti-miscegenation laws that said it was wrong for a person who was a white person, a, a man to be in a relationship with an Aboriginal woman.
So it was basically an apartheid type of law which existed in Australia. And today we would look at that and we would say, “That was wrong. That was an unjust law.” And there’ve been other laws of that kind around things like slavery. And it’s [00:21:00] important here to understand that there were people at the time who knew that that was wrong, and nonetheless, these things persisted because they had that status of law.
So it’s a contingent fact about the world, if you like, that such laws exist and they can be amended, they can be repealed, and so forth. Now, below that, the reason that we’re able to say that something is an unjust or a just law is that there’s some kind of other standard which seems to be more permanent that we can appeal to When you ask a person what should one do, okay, including what laws should we have, typically what they would go to is to a moral framework.
And so the overarching question about what should one do is answered in multiple ways throughout history by people who present to communities and to individuals little packages which have got, if you like, uh… Imagine a, a little box [00:22:00] which has got the label morality on it. You open up the box, and inside you can see some values, some principles.
In the case of religions, revealed truths, exemplary lives of saints, of gurus, and others. And somebody says when– to, to a person who’s just recently been born into a community, they say, “Now, when you get presented with the question what ought one to do, here’s the box. Look in the box. This will tell you what you ought to do.”
Now, the thing about a moral life is that it is possible to be inducted into that, to learn how to apply those values and principles without ever really questioning it. Mm. And in fact, one can become habitually moral. So you might be habitually kind, habitually truthful. Why? Because that’s the box I go to.
So we’ve now got law, morality, and then we come to the deeper level, and that is ethics. In ethics, particularly in its expression in the West going back to Socrates, [00:23:00] is by definition an examined life. It goes back to Socrates’ claim that the unexamined life is not worth living. So although you may have the same values and principles as a person who’s living the purely moral life, if you’re living an ethical life, then what you will do is you will stop and you will reflect.
You will think about the circumstances in which you apply. You will ask yourself whether these values and these principles are the ones that ought to be applied. In fact, what you will have done is you will have taken something that was derivative, that is merely inherited from your culture or your religion, and you will have made it your own.
And kind of the, the psychological story of the difference is that when you’re very young and you’re growing up and somebody says, “Well, what should we do?” The voice you hear in your head is that of your mum, your dad, or some community leader or influential figure And then when you reach that point of ethical maturity, even though the answers might be the same, it’s [00:24:00] your voice that you hear.
Ah, I understand it. But it’s essentially the examined life. And the claim made by philosophers, at least I would certainly make it, and I’d be in good company, is that this is the kind of life which is fit for the form of being in which we participate, namely human being. Because unlike other animals, we’re definitely animals and we have instinct and desires, but we’re not, we’re not kind of captured by them in the same way.
We can go beyond what they might drive us to do and make conscious choices.
Johanna Weaver: Mm. And so what I love about that description is, I think most commonly when I… at least from a lawyer’s perspective, when people talk about law, moral, and ethics, it’s in a hierarchy. So people would– well, lawyers put law at the top.
And what I love about this is you’re kind of flipping it. Sorry, I say you are, but, uh, a long, uh, tradition of philosophy . A long tradition, yeah. When we look at governing AI, there is a much deeper tradition of ethics around [00:25:00] AI than there is around hard law and regulation. What do you say in defense of ethics when people say that ethics aren’t enough?
The artificial intelligence that we’re currently experiencing, there are ethical frameworks that are in place, but the, um, artificial intelligence that we’re experiencing in society isn’t ethical.
Simon Longstaff: So I would never say that we should not have any laws. What I worry about is that, this goes right back to Aristotle, that- You can be in a world where many, many things are going wrong, not because of any malicious intent on part of any individual, but because good people are making bad decisions.
And I’ve spent over thirty– around thirty-five years now talking to people, particularly where that’s happened. And when you ask people to tell you about how they came to be responsible for making that choice, they say with total sincerity, “Well, I, I just didn’t [00:26:00] see it at the time.” They can see it now, but they couldn’t.
And why couldn’t they see it? The most common answer is because everybody was doing it, or that’s just the way it was done. But one subspecies of that are people who say, “When I was thinking about it, I was told if it’s not illegal, it’s not wrong.” Now, that move to say, “Well, if it’s not illegal, it’s not wrong,” is effectively to invite a kind of conditioned blindness where you learn not to look at a richer range of considerations because they simply fall outside the boundary of what the law contemplates.
So I would want to have a world in which there is a strong ethos, that is ethical foundation, in which people are constantly wondering about why we’re doing this. Is that a legitimate purpose? Uh, what are the values and principles that really ought to apply for us at this time? Uh, they may be very stable, but they’d be constantly being thought about and reviewed so that you’re looking around.
You’re not, you’re not [00:27:00] being blind to certain dimensions of the world which are simply you’re oblivious to, and therefore you find yourself, it’s like walking in a fog over a kind of edge of a cliff into an ethical death trap. And given that work, then I would say, of course, we want to use the resources of the law to be able to set some hard boundaries where necessary, and we want to have the same approach to conscious institutional design to make sure that that’s there.
But I would say let’s not Think about deploying things of great power and utility in conditions of blindness where we just don’t see what we’re doing. And the best antidote I know to seeing more is what ethics has to offer.
Johanna Weaver: Yeah, I agree, and but I didn’t before I met you, Simon. So there you go. You’ve got to convert- Oh, really?
Yeah, exactly. Oh. Because I think, you know, so many times I’ve heard people say, “Well, ask the lawyers. Is it legal?” And the answer comes back, and [00:28:00] that, that can’t be the end of the conversation. You mentioned also earlier about affordance. Another A word that is used so often when we’re talking about artificial intelligence is accountability, which is not quite…
It’s not the same as affordance. But when we’re talking about AI systems that are becoming more and more capable, what affordances do you think are minimum affordances that we need to build into systems? Um, and then how do you see the difference between affordance and accountability?
Simon Longstaff: Okay. So I’ve already mentioned some of the affordances that I think we have, of which the most important is the ability to monitor, whether you’re using blockchain or other systems for this, the extent to which an AI stack continues to preserve the original design intent in terms of its ethical content, its [00:29:00] purpose, its values, and principles.
Of course, I’m presuming in this answer that we have people who are building these systems who understand precisely what I’m getting at and the distinction between those three things, and that in their markdown files it’s all there, and it’s published, and they’ve had the ability. Which I know, because I’ve done the work on it, that it’s technically possible to give these things effect.
There’s no point if what you get is a kind of subtle mutation in the underlying code that takes it away from those express purposes, values, and principles. So a disinterested but expert third party has got to be able to audit this and see that it’s still preserved. That’s a critical affordance. Now, that then, though, runs into the roadblock, which brings about the difference between affordances and accountability, and that is the fact that although we can explain what was intended and we can explain what we sought to bring about, and I’m talking now about particular large language models [00:30:00] and neural nets, things of that kind, what we can’t explain is precisely then what happened inside the black box.
And because we can’t do that, accountability in one sense is almost impossible, maybe literally impossible. And so what we need to do is to rely upon what I describe as necessary fictions of a kind that we’ve had to do in other places. So you take a, a liberal demo– democratic society like Australia. We have still intact, although somewhat weakened, we rely on the doctrine of ministerial responsibility, which is based on a necessary fiction.
And that is that a minister in the executive actually knows about and is accountable to the parliament for everything that is done in a department, no matter how sprawling and massive and complex it is. We know that it is [00:31:00] impossible for any single individual to know every decision, every– to see within every nook and cranny.
But it’s essential to our democratic system that we pretend that they do. And anybody who volunteers to be a minister, when they put up their hand and say, “I will accept that commission,” they should know that there is that necessary fiction that despite it being literally impossible, nonetheless, the conventions require that we pretend that it is true that they knew, so that when they stand up before parliament, they can be held accountable in that sense.
I, I think we need the equivalent necessary fiction in relation to AI, where at least with the state of the technology as it is now in the most powerful models We know that nobody, not even the person who created the system, can actually give you a step-by-step account of what has happened inside the blackboard.
It’s not like Boolean logic where A leads to B leads to C and so forth. [00:32:00] But it’s a necessary fiction that the people in charge of it must nonetheless be accountable. They are responsible in that sense, even though they c- they, they can’t literally know for the operation of these systems. And that’s where I think we’ve yet to map out how that actually works.
I don’t think we’ve even agreed on the fact that necessary fiction is required, but that’s what I think this system, as the technology is at as of today, it requires, and without that, we can’t have any accountability.
Johanna Weaver: Yeah. I definitely don’t think we’ve reached the point of agreeing that fiction is required.
It’s persuasive that it is. I think all of the conversations that I’m having are about the fact that actually that cause of… the chain of causation can’t be established, therefore we’re in a challenging situation. So I, I’ve, I like that framing.
Simon Longstaff: Nobody who is in charge of these systems is required to do so other than because they chose to place themselves in that position.
And, and, and you’ve got to understand that when you make that choice, it [00:33:00] will almost certainly, if not now, sometime in the future, require the invocation of that necessary fiction, and you will be held accountable for what is done, even though you cannot know. Mm. And pleading that as a defense won’t be adequate.
Johanna Weaver: Yeah. I can certainly see that coming down the future, and I think that the expectations of the public in our new social compact that we were talking about will lean towards that, and this is a big part of how having people engaged. And one of the things we try and do in TechMirror is to say to people, “Get engaged in these conversations.
Tell your politicians that you want this type of accountability, um, because these are the decisions that are being made right now.” Uh, Simon, a- another thing that I always really enjoy talking about with you is what AI is forcing us to learn about what it is to be human. You have said, for example, that it’s– AI is not just a technological revolution, but that it’s a mirror forcing us to rediscover what it means to be human.
And [00:34:00] given we’re on the Tech Mirror podcast, which is actually a play on exactly that concept, I couldn’t let that go, uh, without asking you. So what do you think that maybe we’ve forgotten about being human, that AI is now making impossible to ignore? Or, or is that… I saw you frowning there. Maybe challenge the- No, no, I- Challenge the premise of the question.
Simon Longstaff: No, no, I was just going to say, you’re just a remarkably good listener. Most of what I say, people just completely gloss away. No one ever remembers it. I was, I was feeling very impressed by the fact that you remembered. It– Look, it’s, it is far more than technical. I mean, people- Yeah … it’s a commonplace to talk about it being a sociotechnical revolution, and it is a revolution in that sense.
But the point I’ve made is that it comes back to some work I was doing for orthopedic surgeons around the world, where, uh, I was asked in a number of different places, here, the US, and later in New Zealand, to talk about the end of the surgeon. Because, I mean, it’s not science fiction to imagine a world in which a surgeon walks into [00:35:00] an operating theater and draped over the anesthetized body of a human is a digital twin.
And well, I’m, I’m taking a step. You, you might get all of the cues with a augmented reality to know where to cut, how deep to cut, a whole lot of things. And then somebody says, “Well, why do we need the human in the loop? Surely, if you’re a person who’s having an operation to remove a cancer, you want something which has got machine-grade vision that can see beyond the spectrum that we can recognize.
It’s got the precision and the sight to be able to distinguish between healthy flesh and diseased flesh, so that only the minimum is taken with the slightest.” And say, “Well, why would you– Where, where’s the human in this?” And that’s a kind of a paradigm case, where- That sort of question arises just as it does in something, for example, in terms of diagnostics, where already AI is superior in its ability to look at pathology and histology and [00:36:00] identify a particular disease condition.
Now, the difference is this. There is no machine on earth, no robot, no AI stack that can be in possession of the knowledge say that you, the patient, only has a matter of months to live. Who can tell you this in a way that another human being can, where they, they reach out and they have a consoling touch on your shoulder, and they say, “I’ve got some terrible news.”
And as they tell you that news, you know that they know what it means to be mortal. That is to contemplate the prospect of an unsought for and early death and the kind of empathy. Um, I was in another discussion recently where there was a discussion about the use of AI to improve safety conditions in a, in a really dangerous working environment.
Where of course you could have cameras and sensors and automated [00:37:00] systems that could stop- An at work, at-risk worker from putting their hand, say, into a dangerous conveyor belt, and it can just stop it on there and say, “Down, down,” you know, flashing lights, stop, danger, danger. But that is so different from another human being who says, “Stop.
Watch out,” where they convey to you a sense of their care for you. Now, these are distinctively human attributes, and, and the question that I’ve had, really, it, it started to come to my mind when I saw people writing on menus that they were hand-cut chips. I thought, “God, why would you wanna tell me that the potatoes have been cut by hand?”
And obviously there’s a sense that the person who’s making the chips and selling the chips thinks you’re gonna value this, that you would rather have a hand-cut chip than a chip cut by a machine. This of course predated the emergence of artisanal breweries and bakeries and all this. There’s something about that which bears the mark [00:38:00] of its maker which is valued.
There’s something about the human sense of empathy, about knowledge of our mortality. So they’re just three examples of things which you– excellence, the technical excellence of what the machine can do forces you back to say, ‘Well, what do I bring that’s distinctive about my humanity that simply can’t be replicated?’
Johanna Weaver: I love that you’ve managed to link in there mortality, chips, and craft beer all in one, all in one response. And why
Simon Longstaff: wouldn’t one?
Johanna Weaver: Exactly. It’s amazing. So Simon, I’m conscious we’re coming up to time. So I’ve got two, two questions left for you. The first one is about trust. This is something that we’ve been focused on a lot in many different conversations that we’ve been having, but also that it…
that society is having at the moment in terms of, do we trust AI and should we trust AI? I think there’s a very large question about whether there should be a presumption that we want to trust AI or whether not trusting it in and of itself is a good thing. From your perspective, what [00:39:00] are the conditions for trust?
How do you build trust, uh, specifically looking at it in the context of AI?
Simon Longstaff: Two things are required for trust. Firstly, you must declare, that is the person, if you desire to be trusted, you must declare the standard by which you are prepared to be judged, and that standard will be in the form of your values and principles.
Say, “These are the things. If you watch me, these are the choices I will make. These are the good things I will pursue by the means conditioned by the principles which I declare.” That’s the first step, because if you don’t declare that upfront, no one can know how to judge what you do. It’ll just be what
There’s no, there’s no point of calibration. There’s no way of seeing how far you deviate from or accord with some standard for judgment. So it’s essential you declare the core values and principles which are going to be the basis for which you are prepared to be judged. Now, of course, the second thing is the extent to which you [00:40:00] act in accordance with those things.
So they’re the two obvious parts. Then there’s the third part, which is often overlooked, in that people have not for a long time,
if ever, but certainly not for a long time, simply judge whether a person can be trusted as a result of their own direct experience. So Johanna, if you said to me, “These are my values and principles, and Simon, I’m gonna treat you like this,” and if 100% of the time that’s perfectly aligned, then I would say, “That’s all well and good, but how do you treat other people?”
People who are not me. So for corporations, for example, they might think if they’re a big retailer, for example, and say, “Oh, we’ve told people who we are and we, and we treat our customers like this every day. We’re relentless.” But they need to realize that their customer’s gonna be asking, “Well, well that’s very good, but how do you trust e- how do you treat each other inside that company?
How do you treat your suppliers?” Because if we can see a gap between what you say [00:41:00] you stand for and how you treat anybody, then because we’re skeptical and sometimes even we’re, we’re cynical, we’ll say, “Aha, there’s the exception that proves the rule.” So it’s no easy thing to merit the trust of others because you’ve got to be clear about that standard, but then you’ve got to be consistent in the way you apply those values and principles across the board.
And of course, there can be tensions within these values and principles. I wish I could say we lived in a universe where everything that’s good lined up and there was never a tension. The truth is that most of the challenging ethical questions we face in our lives are not between good and bad, right and wrong.
It’s good versus good. It’s right versus right. Uh, that, that’s… And that’s why people don’t like ethics so much, so much prefer the self kind of the certainty that a fundamentalist of one kind or another, political, religious, or otherwise can give or to escape into a kind of a hedonistic haze and hope that someone else sorts it out.
But if you’re prepared to stay in that space, it is messy. But [00:42:00] that means then that if you are as a person wanting to be trusted, perhaps having to compromise between truth and compassion or some other set of competing, you should be open about that. You shouldn’t pretend, “Oh well, you know, this is a good outcome, so it excuses me doing that.”
You could say, “No, actually, I, I wrestle with this and I violated one of my own values, the ones I’ve told you about. And I– But this is how, why I did it. And, and this is the way I tried to preserve that one to the greatest extent possible.” And typically, in public life, we don’t hear those conversations. We hear people saying more often, “Oh well, I just made this as the right choice.
Ultimately, I’ll be proven to be correct by some index of judgment.” And so we lose some of that fine granularity that we want. And in losing that, that’s how trust begins to ebb away because people say, “Oh, you didn’t, you didn’t notice that you violated one of your own values or principles,” or, “You didn’t feel like you had to tell me why,” or, “You didn’t say how you balanced it.”
And people say, “Well, you’re a hypocrite and I can’t trust you.”
Johanna Weaver: Yeah, and I think you see that really [00:43:00] prevalent in many of the conversations where you have tech companies who say things like, “No, regulate us. We wanna be regulated.” And then their actions are quite different or they’ll push back on enforcement.
And so I think when you’re describing it as these are the standards we live by and then actually living, living by those stand– demonstrating that they are actually putting those standards, walking the walk, that’s often I think where there is this mismatch in trust with, with large companies.
Simon Longstaff: And it’s not just that they say one thing and then say– do something explicitly which is at odds with that.
What they also do, without necessarily realizing this, is that they build systems, policies, and structures which convey messages at, that are completely at odds with what they say. So we’ve seen this, for example, in banking and financial services for a period of time where they said, “Oh, well, customers are really important.”
And yet all of their KPIs had nothing to [00:44:00] do with the wellbeing of the customer. It was all about profitability. And people said, “Well, if you say that but you’ve created systems which drive people to do something else, then you are hypocrites.” And that charge of hypocrisy is lethal. It just Creates cynicism.
It, it destroys the bonds of association and community. And so often it’s not a product of, uh, any kind of deliberate intent on the part of people who are CEOs or otherwise. They just don’t think. They’re just part of that state of conditioned blindness we talked about before, and they destroy trust. And of course, just on your point about them saying we like regulation, it’s really interesting.
I’ve seen this now over a long period of time. If you give a choice to a company between voluntarily choosing to apply a high ethical standard or instead being forced to apply that standard by regulation, so often they will say, “Please regulate me,” because they [00:45:00] either crave some kind of illusory sense of certainty or they think it will force their competitors to adopt the same standard because they have this hunch that perhaps if they were to adopt the higher standard, it may actually work against them in a competitive marketplace.
When in fact there’s as much evidence to show that in fact it could be to their advantage.
Johanna Weaver: So Simon, I, I think that’s absolutely right in terms of what you’re saying of the power of the potential commercial benefit that could come from companies that recognize that actually designing and making ethical AI is a competitive advantage, and you’re starting to see that in the market.
Not, not a lot, but you’re starting to see that, and that
Simon Longstaff: makes me- Well, I think the story of Anthropic recently is possibly an example of that. Yeah. I hope it turns out to be a good story, but- It
Johanna Weaver: could, it could go either way . So,
Simon Longstaff: yeah.
Johanna Weaver: But, but let’s end on… Let’s keep that positive tone to end on for us. Um, can I ask you to help us to paint a picture of what the world could look like in 10 years’ [00:46:00] time if we have more people waking up to the power of, well, the fact that this is not, technology is not evolving in any particular way inevitably, that it can e-evolve in many different ways, that we have the power to shape it.
Um, but also the power of ethics to help us to frame the way that AI may be unfolding into our society. What could the world look like in a positive way?
Simon Longstaff: Okay, so let’s go back to ancient Athens and Aristotle’s view that ethics and politics were different sides of the same coin. Uh, one being the good life for the individual and the other, the good life for a whole community or society.
Now, I think what we should do is to think in those terms of not just as a technical question to be resolved or informed by ethics, but as a much larger system about the kind of society [00:47:00] we want to be. So we’re gonna have to think about a world which I’m really optimistic about what’s possible, in which all of the dangerous, dirty, and- Drudge, drudge-like work is taken over by systems that can do it without any exposure to pain or distress, and it alleviates us of those burdens in a way that has not been true in the past.
But then at the same time, which causes us to ask, well, what is a good life for a person living in such a society? And what are the economic and political adjustments, and they’re gonna be profound, that we need to make in order to sustain such a society in which everybody can flourish? I would love to live in, in that world where all that is beneficial can be harnessed with the risks well contained, and where it serves the purpose of lifting up a society rather than [00:48:00] simply enslaving it to a particular model or making human beings commodities who work in service of the machine.
We’ve seen Fritz Lang’s film Metropolis. Uh, if anybody’s seen that, that’s the vision. It’s actually an incredibly dystopian view of what technology could become if it’s given or its logic is allowed to unfold in a way that’s indifferent to the wellbeing of people. And I’d encourage anybody to look at that just to see what it anticipates could happen.
I don’t think that is in any sense inevitable. I think it could be a brilliant future, uh, particularly for a country like Australia, which has got all the natural resources and the people and the institutional arrangements to create an extraordinary period of– in which it’s a, could be the most just and prosperous society the world’s ever known.
And in doing that, it could be aided by a combination of AI, robotics, and other things, so it can realize that potential.
Johanna Weaver: And if you could make– If you could have a magic wand and change one thing about the way people think about AI [00:49:00] and ethics to help make that future come true, what would it be, Simon?
Simon Longstaff: It would probably to get people to be curious about why things are done.
Un-unleash your curiosity and ask, is what’s being proposed in a form which allows you to identify the purpose which is going to be served? Is it its purpose that you can approve? And then does the value– Do the values and principles which they’re saying they’re trying to embody align with yours? And if not, what are you going to do about it?
Rather than washing your hands and being indifferent. And it may simply be just what small purchase you make, where you exercise a little bit of economic sovereignty, or it could be the way you vote or the letter you write to your MP or the fact that you go to a meeting or attend something of interest.
Any of those things can collectively make a big difference.
Johanna Weaver: Simon, I, um, I always note when I get emails from you this, that you have in your signature block that, “An unexamined life is not worth living,” from Socrates, which you’ve referred to many times. But, but also thank you so much for [00:50:00] challeng- challenging us to also think about that and to apply those concepts.
I’ve really enjoyed this conversation. Uh, thank you for joining us and, uh, we hope to talk to you again sometime soon.
Simon Longstaff: I’ve really enjoyed it too, and thanks for having me on.
Johanna Weaver: Thanks, Simon
Well, that’s it for this episode of Tech Mirror, which is brought to you by the Tech Policy Design Institute. We’re based here in Canberra on the lands of the Ngunnawal Ngambri people. If you found today’s conversation useful or thought-provoking, please do share it with a friend or a colleague, or leave a review and subscribe wherever you get your podcasts.
If you’re watching, please don’t forget to like and follow. For show notes, you can visit techpolicy.au/podcast. And this podcast was made possible with thanks to the generous contributions from government, industry, and philanthropy to the Tech Policy Design Fund, the full details of which are [00:51:00] available on our website.
The team at Audiocraft produced this pod on the lands of the Gadigal people of the Eora Nation. Music is by Thalia Scopellitis. A big thank you also to the team at the Tech Policy Design Institute, without whom this pod would not be possible. Thank you for joining us, and as always, get in touch and get involved