“AI Is More Dangerous Than Nuclear Weapons” (From a 27-Year Air Force Insider)
Summary
After 27 years inside the French Air Force working on the ethics of drone and autonomous weapons warfare, Dr. Emmanuel Goffi witnessed the shift from battlefield decision-making to remote killing, and now to a world where artificial intelligence may shape reality itself.
In this episode, he explains what lethal drone strikes actually look like behind the scenes, how responsibility moves away from political leaders and onto operators, and how militaries carefully construct the narratives that make controversial technologies feel acceptable over time.
We also explore Western dominance over AI ethics, the risks of exporting a single moral framework to the rest of the world, and how deepfakes and algorithmic media threaten public trust. At one point, Goffi delivers his starkest warning: “Narrative being shaped by AI is a bigger threat than nuclear weapons.” The next conflict, he argues, may not be fought over territory, but over perception itself.
Guest bio: Dr. Emmanuel Goffi is a philosopher of artificial intelligence and a former French Air Force officer who spent 27 years working on the ethics of drone and autonomous weapons warfare. Now co-founder and co-director of the Global AI Ethics Institute in Paris, he studies how emerging technologies are reshaping responsibility, geopolitics, and truth itself. His work focuses on the ethical consequences of AI, and why the future of conflict may be decided as much by narrative as by weapons.
Top Takeaways
Military Drone Warfare: How Governments Gradually Normalize Killing From a Distance
Dr. Emmanuel Goffi describes how military drones went from controversial technology to an accepted part of modern warfare in less than two decades. During his time in the French Air Force, he helped develop language intended to reassure the public about drones, even as the military anticipated eventually using armed versions. Goffi argues that repeated messaging, combined with gradual technological changes, can make once-unacceptable practices seem ordinary. The larger concern is how easily public acceptance can evolve without a corresponding examination of the ethical consequences.
Who Is Responsible for AI Drone Strikes? The Problem With Blaming the Operator
One of Goffi's central concerns is how responsibility gets distributed when military drones kill civilians or make targeting mistakes. Although political leaders authorize military operations and numerous people contribute to targeting decisions, he argues that responsibility often gets pushed toward the operator who ultimately pulls the trigger. AI targeting systems complicate matters further by introducing software developers, private contractors, and algorithmic assessments into the decision-making process. Goffi believes accountability should be determined by examining everyone's involvement after an incident, rather than deciding beforehand who will take the blame.
Military Propaganda and Political Messaging: Why Generals Should Be Questioned Like Politicians
Drawing on his experience working with the French Ministry of Defense's communications apparatus, Goffi argues that statements from senior military officials should be examined as carefully as political speeches. Generals and other high-ranking officers are not simply presenting neutral assessments. They may also be trying to shape public opinion, justify military decisions, and build support for particular policies. Our discussion compares these performances to professional wrestling, where public confrontation can conceal considerably more cooperative relationships behind the scenes. The takeaway is not that every statement is dishonest, but that audiences should distinguish strategic communication from independent analysis.
The Military-Industrial Complex: When Tech Companies and Governments Share an Interest in War
The conversation explores the increasingly interconnected interests of technology companies, military institutions, and political leaders. Goffi describes an environment in which national security objectives and commercial incentives can reinforce each other, potentially encouraging military expansion or conflict. I raise the concern that AI weapons companies might benefit not only financially from war, but also technologically through access to battlefield data. Goffi does not establish that companies deliberately initiate wars to train AI. Instead, he warns about the broader ecosystem of overlapping incentives that can make ethical objections less influential.
Western AI Ethics: Are the United States and Europe Imposing Their Values on the World?
Goffi challenges the assumption that Western ethical principles should automatically govern artificial intelligence everywhere. He argues that Western approaches emphasizing individual rights, privacy, equality, and universal principles can conflict with cultural traditions that place greater importance on collective responsibility, religious authority, or different understandings of personhood. His criticism focuses particularly on AI governance frameworks developed in the United States and European Union that may become international defaults. The discussion raises a difficult question: when a small group of influential countries establishes the ethical rules for a global technology, whose values are being represented, and whose are being overlooked?
AI Ethics in Africa: Why Inclusion Isn't the Same as Genuine Collaboration
Goffi argues that Western initiatives to include African countries in AI ethics discussions can unintentionally reinforce existing power imbalances. Inviting people into a framework that has already been designed around Western assumptions is different from allowing them to help establish its foundations. We also discuss how international diplomats and researchers working in Western institutions may not adequately represent the full diversity of perspectives within their home countries. Goffi advocates a more collaborative approach in which different cultural traditions can genuinely influence how AI systems are developed and governed.
Ubuntu, Buddhism, and Māori Traditions: How Different Cultures Could Change AI Ethics
The discussion examines how non-Western philosophies might fundamentally change our relationship with artificial intelligence. Goffi introduces African Ubuntu traditions, Japanese Shinto beliefs, Buddhist approaches to identity and ownership, and Māori perspectives in New Zealand. He explores how traditions that emphasize interconnectedness or spiritual relationships with the natural and material world might approach questions about robot rights, privacy, and data ownership differently. Goffi points to New Zealand's engagement with Māori perspectives as an example of cultural knowledge influencing technology governance. His argument is not that these traditions necessarily offer superior solutions, but that excluding them limits the ethical questions we are capable of asking.
AI Augmentation and Human Enhancement: A Future Divided by Technology
Looking toward the future, Goffi discusses a world where augmented reality, implanted devices, and AI-assisted systems could increasingly blur distinctions between physical and digital experience. He anticipates new inequalities between people who can access technological enhancements and those who cannot, particularly when financial resources determine who benefits. I also raise the possibility of a philosophical divide between people who embrace augmentation and those who deliberately reject it. The broader concern is that technological advancement may create not just economic disparities, but fundamentally different ways of experiencing reality.
AI Deepfakes and Synthetic Media: When Seeing Is No Longer Believing
Goffi warns that increasingly convincing AI-generated videos, images, and voices are making traditional methods of judging credibility less reliable. We discuss a reported deepfake depicting a military coup in France, along with scams involving cloned voices and impersonated relatives. As synthetic content becomes more convincing and widespread, he worries that people will increasingly mistake generated material for authentic evidence. The conversation also explores the risk that false information circulating online could influence future AI-generated answers. Goffi's proposed response is an older intellectual discipline made newly urgent: verify information through independent sources, even when it appears convincing.
AI Narratives vs. Nuclear Weapons: Why Goffi Fears Control Over Information More Than Military Force
Perhaps the episode's most provocative argument is Goffi's belief that AI-generated narratives could become more threatening than nuclear weapons. His reasoning centers on the difference between physical destruction and psychological influence. Nuclear attacks carry the threat of retaliation, while the manipulation of information can spread quietly, cross borders, and shape beliefs without an obvious moment of confrontation. Goffi argues that controlling how people interpret reality may become an extraordinarily powerful geopolitical weapon. We end by considering how individuals can practice methodological doubt, questioning and checking information without allowing skepticism to become indiscriminate distrust.
Top 10 Quotes
Quotes have been lightly edited for clarity, removing verbal fillers and repetitions while preserving the speakers' original wording and meaning.
Military Propaganda: Generals Are Politicians in Uniform
"High-ranking military officers, like generals or colonels, are political leaders with a uniform. They're trying to influence people. It's all about critical thinking. You must always doubt."
— Dr. Emmanuel Goffi
Drone Warfare and Accountability: Why Operators Become Scapegoats
"Factually speaking, the chain starts basically at who pulls the trigger, because what you want is a scapegoat. You don't want to go back all the way and find who is responsible for what."
— Dr. Emmanuel Goffi
Military Ethics: When the Decision Has Already Been Made
"Once the agenda is set, I feel like there is nothing that we can do against that."
— Dr. Emmanuel Goffi
Western AI Ethics: The Danger of Believing Your Values Are Universal
"It's always weird to see that we feel like we hold a truth, that our values are universal, and that we are entitled to spread them and even to impose them to the rest of the world."
— Dr. Emmanuel Goffi
Global AI Governance: Why Inclusion Can Reinforce Inequality
"Including means that I am superior and you will just come into my moral setting or my moral environment."
— Dr. Emmanuel Goffi
Understanding Other Cultures: Perception Isn't Knowledge
"We have perceptions about the world, but we don't have knowledge about the world."
— Dr. Emmanuel Goffi
AI Deepfakes: Why We Must Question What We See
"You have to doubt everything, even your own senses. When you see something, you cannot trust your eyes."
— Dr. Emmanuel Goffi
AI and Nuclear Weapons: The Power of Controlling the Narrative
"Words are really weapons of mass destruction. So if you have a system like AI that is able to spread a narrative widely at a very fast pace, then you master the world."
— Dr. Emmanuel Goffi
Artificial Intelligence: How a Name Changes Our Perception of Technology
"The fact that we feel like there is a threat is because we are using the word intelligence, and because we have our intelligence, and then we have competition between two intelligences."
— Dr. Emmanuel Goffi
Critical Thinking in the Age of AI: The Importance of Methodological Doubt
"There's a method for doubting. It's not about being skeptical about everything, because you would turn crazy, and that doesn't really make sense."
— Dr. Emmanuel Goffi
Full Transcript
Zach: There's a frightening question we must now confront with modern warfare. If machines and algorithms decide who dies, who is actually responsible? Is it the drone operator, the commander, the software engineers, or the deep state strategists crafting the stories that made the strikes feel acceptable in the first place? My guest today, Dr. Emmanuel Goffi spent 27 years inside the French Air Force working on the ethics of drone and autonomous weapons warfare right at a time when the military was getting good at killing from a distance and even better at making it seem necessary to the public.
He saw firsthand how the narrative around drones was carefully shaped, how the language of warfare gets crafted inside think tanks as they get ready to sell it to the people, and how generals often operate less like battlefield commanders and more like politicians in military uniform.
Emmanuel: Every time you have kind of a discourse or narrative that is provided by a high-ranking military, it's like having a discourse that is provided by a politician because high-ranking militaries like generals or colonel. There are political leaders with a uniform. They're trying to influence people. You know, it's all about critical thinking. You must always doubt.
Zach: Dr. Goffi witnessed the shift from a time where people genuinely believed we'll never have armed drones to a world where now remote killing is the new normal. Once autonomy enters warfare, ethics doesn't just blur, it mutates. After his career in the military, Dr. Goffi moved from drone strike ethics to the much more ominous question he works on today. What happens when artificial intelligence starts shaping not just weapons, but the truth itself? In this episode, we get into what lethal drone strike decision-making actually looks like behind the scenes.
How civilian harm gets discussed differently inside war rooms than it does to the public. why the burden of killing keeps falling on drone operators rather than the leaders who authorize the strikes and how modern deepfakes are disintegrating the public's trust in reality. We also talk about how the West is secretly shaping the moral DNA of artificial intelligence and what happens when just a handful of countries is writing the default operating system for the most powerful technology on the planet. At one point, Dr. Goffi told me that the only thing he fears more than nuclear weapons is AI.
Emmanuel: Words are really weapons of mass destruction. So if you have a system like AI that is able to spread a narrative widely at a very fast pace, then you master the world. And I feel my take now is that this narrative that is conveyed by artificial intelligence is a bigger threat than nuclear weapons.
Zach: If AI starts deciding what we see, what we believe, and eventually what we accept as reality, the biggest battle of the future won't be for territory, but for our perceptions. Whoever can sink their hooks most deeply into our minds to control the narrative wields more power than any arsenal of nukes. Let's get into it. Dr. Emmanuel Goffi, thank you for joining me on the show. I appreciate it.
Emmanuel: Pleasure. Thanks for having me.
Zach: So you you've served in the French Air Force for 27 years, which is a long time, where much of your work, as I understand it, involved ethics applied to drones and autonomous weapons. I'm curious, how do how do you get into that line of work? Like what drew you into it initially?
Emmanuel: Actually what happens is that I resumed my studies when I was in the military as an officer and I had a class on ethics applied to international relations and when I came back after my two years master degree in the air force. It was exactly just after 9/11 and we were considering lots of questions around drones and the renewal of our own fleet in France and then I've been asked to work on that because I had this kind of background in ethics and also in laws. So I I've been asked just to work on that for a while.
Zach: Was there something in the class some some case study or something the teacher said that made you want to continue in that direction? cuz I I've taken a lot of classes in college where things just did not stick. You know, I ended up being a business major in accounting, but there were some some teachers just have a way of captivating you. So, I'm curious if there was anything that stuck out to you in that class that made you just double down in that direction.
Emmanuel: I think what I what I really liked that two things. The first one is that when you're in the military, actually the point is to narrow your perspective, right? Because you have to obey orders, right? So they do not open your mind. That's the contrary, right? And when I when I ended up at Sciences Po in Paris, it was like, okay, they opened up the world to me saying, okay, look at all the nuances.
And the class that was actually taught by [name unclear] was this kind of class where the world was made of nuances and he has this kind of crazy ability to teach and to share things in a way that makes it really appealing interesting and you want to dive into his world. And especially because when we were doing that, we were mostly focusing on military matters because when we're talking about international relations and when we're talking about ethics applied to our international relations, it's mostly about the use of military force this kind of thing.
And first I think is just a great teacher is still teaching at and second the fact that he was opening my mind to the world in a way that I've never experienced before. That was just like a shock, a real new experience to me.
Zach: Yeah. walking walking through the world thinking about the application of ethics is something that I guess for lack of a better word civilians like myself where I think about a situation and if I'm brought to think deliberately about ethics or if I'm maybe looking at a case study for fun online I'll I'll think about oh wow like the ethical applications of something like the production of iPhones overseas or the ethical applications of tourism and going to more developing countries and spending my but like it doesn't really enter my mind unless there's an outside an outside spark from something where with you I imagine because it's your job it's a it's a really enlightened but also it seems like kind of a scary way to walk through the world where you're hyper sensitive to ethics all the time where the average person doesn't really think about it.
Emmanuel: Yeah, that's right.
And I actually it was I was sensitive to that because I was in the military and also I could you know I've experienced the military from the inside and I was also in classes where people were younger than myself and they were all into the they have not experienced anything right and it was kind of also really fascinating to see the gap between the way they were understanding the world outside and my understanding from the inside of this kind of world right and navigating this world when you're doing ethics is kind of complex if you have a really narrow perspective on what ethics is but actually ethics is something that is really broad and that sometimes when you're doing ethics I mean for real when you're really diving into ethics basically you understand that if I if I want to simplify that I would say that everything is about ethics right so you do no longer have this kind of problem saying okay I'm doing something that is not ethical which is you know misleading because everything is ethical you just have to find the right way to understand it the right way to analy things.
So, so this is where ethics is really at the same time kind of shaking because sometime you feel like oh if everything is ethical what's the point right I can do whatever I want and at the same time it's really interesting because it helps you dive into yourself right it's really an introspection that you're starting with okay what do I believe what do I think what I'm what I'm thinking and this is the kind of question that you will ask yourself and you will really dive into this ethical world and then you can navigate the even the complex world or the scary world of the military with that.
Zach: I'm I'm curious. So, you mentioned there are a lot of people who they talk about and learn about ethical theories, but they're not involved in real-world practice.
Emmanuel: Yes. Because you know in universities mostly what happens is that for example professors or teachers they're not practitioners, right? So they have kind of the real theoretical background which is great but what they're lacking is the situational you know application of all those theories. So what they are actually sharing is kind of a knowledge but not a know-how. They do not have this kind of you know ability to transpose that into the real-world.
So and basically what you what you're learning when you're doing ethics in universities you're you're looking at books and you're reading Immanuel Kant and you're reading Aristotle and you just try to have illustration that I've taken from the from the news but it's it's it's always really far from your you know your daily life. So you have the feeling that you know while actually you do not know. But when you and I'm sure you understand that when you experience things, your perspective will change. And I've heard things that were totally crazy because people did not really measure what it was to be in the military.
For example, when you're in Afghanistan and you're facing a kid, let's say 16 years old kid, and they say, "Oh, you cannot kill him because that," and they will just, you know, take all the theories showing that it's wrong to arm or kill kids. But when you live it, when you experience it, then you have a totally different perspective and you say, "Okay, that's the theory, but just go on the ground and you'll see how you can apply that." And you will see that sometimes it's not really feasible.
Zach: Is that one of the things that you were working on the ethics of children, combatants, and how to respond to them?
Emmanuel: No, no, no. I was mostly working on the ethics of drones because, you know, the you know, killing from a distance. That was basically the thing that I was doing.
Zach: Yeah. I it's something you made me you made me think about when you brought up the theory is that I guess the practical side of ethics would be policy because you need rules for people to be able to follow to put into effect that become laws and regulations. And I was wondering if you think it's dangerous that there are so many people today that create policies that have severe ethical repercussions that don't actually have experiences in those realms themselves.
There's a lot of people deciding how soldiers should act on a battlefield or deciding whether or not we should go to war or whether you know take over a country. you know even even things like how to run a you know a garbage program in a city like a lot of people are making political decisions without actual experience.
Emmanuel: Yeah. that I feel like it's it's at the same time something that is problematic obviously because they're making decisions that are not always really I would say relevant to the to the to the situation but at the same time it's it's something that is necessary right you cannot ask practitioners to make rules that doesn't work that way so you need to have this kind of policy makers that sometimes are really far away from the know practical things of the everyday life and they have to make decision usually what we expect from decision makers and especially politicians is to ask people that are practitioners to provide them with information, but that's not really the way it works and sometimes it's complex.
But yes, definitely you have rules and I mean in Europe we have tons of rules that have made that have been made by policy makers that are totally totally irrelevant to the everyday life of most companies or actors in the European Union.
And we can see that with artificial intelligence today, right? we have some problems because obviously it will kill innovation and all those people that are making decision at the European Commission for example they have they are not knowledgeable in AI they are not knowledgeable in ethics so they are under the pressure of lobbyist and so they're making decisions that are not always I would say once again relevant at all
Zach: Most podcasts just ask you to listen but the Zach show 2.0 0 lets you step inside the show at the support Zach level. It's the backstage pass. You get early access to every full uncut episode of the Zach Show completely ad-free. You get the Zach Unpack series before anyone else. And you also unlock the Zarives 31 hidden episodes from the early days of the Zach Show that have never been released to the public. If you step into the next tier, Fool's Access, that's where things get personal. You get a private bonus podcast, Zach Gets Real, where I talk about stories I don't tell on the main feed.
You also gain access to exclusive AMAs, my actual prep notes before the interview happens, and a personal handwritten letter from me. Then there's the Fo's Access Supreme level, where the show starts to become not just mine, but ours. You get a heads up who the upcoming guests are, and you can pitch them questions that may shape the conversation itself. We'll also do monthly Zoom hangs and your name appears in the podcast as a producer of the show. No big studios, no executives, just me, you, the guests, and a mic.
If you want to help build something independent and truly be a part of it, come join the Zach Show 2.0 at the zachshow.supcast.com today. That's the zachshow.supcast.com. And thank you for whatever level you choose to support. Now, let's get back to the episode. So, what did the day in the life of someone working on ethics applied to drones and autonomous weapons look like in the military? What problems were you trying to solve on a day-to-day basis?
Emmanuel: The first the first thing was to maybe to dive into those questions. What does that mean to kill from a distance? What does that mean in terms of ethics? Is it acceptable or not acceptable? Or what is the level of acceptability when it comes to using drones first? then to use drones to for example identify a target or even to kill someone or you know bomb someone from a distance. So this kind of questions. So you have to mobilize a lot of theoretical background.
So reading lots of things trying to understand what is the mainstream discourse around that trying to understand where your stance is within this kind of mainstream discourse. And you also have to always think about two things which is something that you can find in the Prince by Machiavelli right it's that you have two different kind of morals that are facing each other. The first one is the moral that you are or the moral the ethics that you that you are selling to the people in terms of communication.
What you're saying to reassure people and make sure that they will not go against you at some point and your ethics as a decision maker in a very complex situation and where the interest of the state, the interest of the nation are way higher than what people are thinking, right? And you always have to work on that. So the work was also to find a way to balance those two things. Making sure that we're not doing wrong things but knowing that sometimes you have to do wrong things right.
You have to what we call fall into the dirty end things. but at the same time trying to make sure that you will reassure the audience and the public just to tell them okay we are not falling into this kind of trap. So basically the job was to understand the overall environment the narrative finding your own stance and try trying to communicate efficiently towards different stakeholders national international etc.
Zach: Yeah. So there there's two sides to it. There's the on the ground side, which I guess would be actually looking at drone strikes and working with operators and decision makers and deciding what an ethical decision would be if you're going to take out targets or you know involve killing people, possibly civilians. There's so many things that can come up ethically that would affect the way you go about it. And then the other side which you mentioned is also the language that you're presenting to the public the selling the public something a technology a war which like I coming from growing up in New York in the United States.
I was very young when the public was being sold the Iraq war. I was probably 10 years old when it happened. And so I learned a lot about that sales campaign after after the fact and how many lies were told and the way that things were sold to the public. So, I'm always very hesitant to believe what I'm hearing from a military communications person that that's that's designed to speak towards the public because I know there's the salesy version of what they're presenting that that's built for people to adapt to new technology over time. That's scary.
But I want to know like what's actually going on behind the scenes as a civilian, which I know is not always possible because of espionage and you want to protect technology and things like that, but I feel like people are much more aware because of the past couple decades or so of like being sold on warfare or being sold on technology.
Emmanuel: Yeah. I think you know I've myself worked with the u the institutional communication of the French ministry of defense. I worked with them for a while and what I can tell you now is that every time you have kind of a discourse or narrative that is provided by a high-ranking military it's like having a discourse that is provided by a politician right because I ranked militaries like generals or colonel basically they are they are political leaders with a uniform no more than that right so what when they're speaking what they're doing is that they are communicating they're trying to influence people so It's not analysis.
You cannot take it for granted and you it's it's you know it's all about critical thinking. You must always doubt. You must always think about what is the purpose of this specific speech. What is the purpose of all those words that have been chosen in a very rigorous way to create a beautiful narrative? Who is behind that? What's the goal of it? why am I the target of that kind of thing?
We are experiencing that in Europe now with the you know the this kind of I would say tropism toward u military matters due to the war in Russia and Ukraine right and I can see this kind of discourse that has that has been reshaped that is recreated here and what I'm used to say is that each time I'm listening to a military or a politician I'm always listening to that with lots of doubt that's for sure
Zach: Yeah, that makes sense, too. Coming from the inside, you know the difference between what's happening on the ground and what's being sold to the public. I heard you mention in a past interview that while you were working on ethics applied to drones in the French military, that was also during the time where the CIA in the US was conducting these drone strikes on Pakistani tribal regions. Can you tell me about that time? because I imagine the French military is interested in a lot of those types of decision makings globally with the CIA.
Emmanuel: Yeah. Yes. For us the big problem was that at the very same time we had a really old fleet of drones, six drones, Arfang drones and we were just considering renewing them. And because of what was happening with the CIA in the tribal zone in Pakistan, lots of people around the world in France included, they were really reluctant toward drones.
They were they were really scary about what drones might become at some point, which is basically little autonomous weapons, right? what we call killer robots. and then we had this kind of situation where we were we were in this need of having new kind of drones but also trying to reassure the people that we were not doing like the CIA that we're not you know doing this kind of strikes illegal strikes because it was it was not yes it was illegal it was out of the laws of war definitely but the big question that we had was about that not that much about the fact that the CIA or the US was striking tribal zone because that was not the big issue.
It was like okay, how can we navigate into this kind of competing narrative about the fact that we need drones but at the same time that drones might be dangerous at some point and then this that was exactly the work that I was doing. were writing what we called language elements just trying to you know explain things in a way that was acceptable to the public telling for example think that now I can feel like that was really communication just trying to explain people that drones were not robots for example because there was this kind of fear of the you know what we call the terminator syndrome that at some point robots that might be drones that might be fitted with AI that might become a threat to human beings right because they might decide by themselves these kind of what we call the terminator syndrome.
So we had element language element saying that drones are not robots which is definitely not true because if a Roomba vacuum cleaner is a robot there is no reason that a drone is not and because there is no clear definition of what robot a robot is. But anyways, we were just trying to make sure that in France at least people were making the difference between what was happening in the US and the strikes in the tribal zone in Pakistan and us not using drones for the same reason but just for civilians.
This kind of thing which was also a lie because we knew that the agenda at the very end of the day would be to have armed drones and we now have armed drones that are operating in the Sahel region.
Zach: Yeah. it seems like if you are if you're trying to sell the drones in a certain way to the public, I imagine it's been surreal watching the evolution of drones since you've been in the military to what they are now. Because when you started, you said they weren't fully autonomous, correct? there were still people operating them. They didn't have the weapons capability that they are now. So when you're selling something to the people, when you're using language to define ethics, it's a it was a much very different game.
And now you have companies like Anduril who make autonomous jets and they can lead flanks of jet fighters and have autonomous weapons capabilities. Drones, submarines. I saw Australia just bought I think it was a billion $1.2 billion dollar worth of autonomous subs with weapons capabilities. What have been your thoughts watching this evolution of the autonomous weapons capabilities of drones in real time?
Emmanuel: I think I I've I've I've I've became a bit cynical about all those communication things and sometimes I'm even questioning the my own practice in ethics saying okay what what's the point with all that because at the very end of the day the agenda is already there right and we'll go where actually policy makers and decision makers wants to go whatever we think about it whatever the discussion that we have whatever the red flag that we can raise saying okay look that might be dangerous maybe we should avoid that Once the agenda is set, I feel like there is nothing that we can do against that.
And it's exactly what I'm feeling. And my feeling is that through communication and political discourses and even high-ranking military speeches, what we're making, we're trying to acculture people to something that is slowly evolving. And while you're a culture to that, you accept more and more things. If I take the case of drones once again in the early 2000 when we had this situation post 9/11 and the question that we had in France against drones I can feel like now we have drones that are delivering armaments in the Sahel region without anyone being concerned by that.
So you see the shift and we're not talking about 50 years or 100 years, right? It's a few years, it's 20 years, not even 20 years, right? And I think that people have become really a culture to this idea of drones to the to the idea of drones being used on the military, you know, battlefield, and even to arm drones and then to drones delivering armaments. And so now it's it's it's the new normal.
And I feel like sometimes the all the communication that has been set by the by the French authorities or even the European authorities or beyond that has been set especially to make sure that people were slowly accultured to the fact that okay now we have drones that are killing and now when I when I'm for example I was watching the news I think that was last week about the new army of robots that were developed by by by China. I feel like people are looking at that like, okay, that's normal. Maybe that's a bit scary, but it's it sounds normal.
While 20 years ago, thinking about killing from a distance through drones was not acceptable. So, you see the shift and I feel like yeah, that that's why sometimes I'm I'm just becoming a little bit cynical about all that, saying, okay, what what's the point with doing ethics?
Zach: Yeah, it I wonder like what is it about human psychology that can allow us to adapt to such severe changes in such a short period of time? Because like you said, it's not over decades. It's not over hundreds of years. It's, you know, a 5 to 10 year shift where you have billions of people who would probably say that like, yeah, we don't want a world where autonomous weapons can go around and taking out targets and have full weapons capabilities and you don't even need a remote pilot. It can just start to make decisions for it.
And you could almost see the next step, which is will there be non-military autonomous weapons drones? Will there be like neighborhood patrol drones for police? Will there be self-defense drones in people's homes that they can buy where, you know, you can watch over your personal property? And to me, that's an example of like something that seems crazy now, but I wouldn't be surprised in 5 to 10 years that there's some company selling you know, personal autonomous weapons, which which sounds insane, but I could see a world in which that happens.
Emmanuel: I think, you know, there is a mix of things that are making that possible. The first one is once again, communication is key. when you're communicating all the time and always having this kind of reassuring narrative at some point people will internalize it. That's first. Second thing is that actually military matters maybe in the US it it's a little bit different. I'm not even sure of that. But in France and Europe military matters is definitely not a concern from for the French people for example they have higher priorities just than thinking about the use of drones and robots.
Sometimes you see that in the news that will last five minutes and say, "Oh, that's totally accept unacceptable. It's scary and the minute after you just switch to something else, right?" Because we're bombarded with information all the time. We are moving from one to another at a really fast pace. So what is the priority for us is mostly what we are living in our everyday life. when you are when you're looking for a job, when you have problems with buying food or paying your rent or this kind of thing, this is your priorities, right?
And the thing that you hear about the use of drones or maybe you know violence in France or whatever or political instability kind of thing, it's it's really short time and then you switch to something else. And plus I guess that when you're talking for example about the use of let's say military tools in the civilian world like you know neighbor I can't remember what you were saying but anyway basically you will not switch from the situ the current situation to that situation. It will be a step-by-step you know move toward what will happen. So once again people will get slowly a culture to the ID.
So maybe you will first have kind of you know robot ducks that will you know wander in the street and that will have a camera on just to make sure that no one is stealing in your house or this kind of thing right and so you say okay that that's that's for me it's it's really good I accept that and then maybe you will have a weapon that will be put on the on the on the dog and that the dog will become something bigger maybe you know a lion and then the lion will become a human being and slowly you will accept things one after the
Zach: The most dangerous type of Pokemon. It just keeps evolving into, you know, bla blastoise with rocket launchers on its back.
Emmanuel: Yes. Exactly.
Zach: Yeah. So, it's like a gradual acceptance over time.
Emmanuel: Yeah. Right.
Zach: I'm I'm I'm curious when you think back to the early mid 2000s, post 9/11, was there a specific drone strike or event from that time that made people inside the French military go like, "Holy [profanity censored in source] this is getting crazy or we need to rethink this." Like a specific strike or sequence that changed the tone for you?
Emmanuel: I No, I don't think so.
We we've heard a lot about once again those strikes that were conducted by the US in the tribal zone and we've heard about strikes against what we called innocent people basically civilians for example in the wedding or these kind of things but I don't feel like people were really concerned about that we had other problems because we were deployed in Afghanistan as you know it and we had the problem in Usbin for example where tens of our militias were killed but not with drones actually with knives It was just crazy. but I feel like people were basically scary about that, not because of specific facts, but because of the mythology that was all around that, which referred to once again the Terminator syndrome and because there was kind of you know, lots of people that were advocating against those system against killer robots.
So we were hearing lots of things about killer robots but that was kind of basic or really superficial perspective a general I would say concern against these new technologies much more than specifically related to a strike or something that happened.
Zach: Yeah. Something that really stood out to me about the autonomous drone strikes and just reading about things happening recently like it there's such a compression of distance with geographical distance, moral distance, emotional distance. You have someone making decisions hundreds possibly thousands of miles away in many cases that have no emotional ties to the outcome. as someone who's applying ethics to lethal drone strikes like how did you evaluate the chain of responsibility because I would I wouldn't even know where to start.
Emmanuel: So I would say that formally speaking it starts with the highest you know political leaders the US president the French president because the hierarchal chain in the military starts start there right factually speaking the chain starts basically at who pull the trigger because what you want is a scapegoat you don't want to go back all the key and to find who is responsible for what and with drones and this new technology with artificial intelligence I feel like this responsibility is way more diluted than it was before.
So now and I remember really the very first discussion that we that I had with for example the campaign kill the robots that was in the early 2000 in Germany in Berlin. They were they were trying to decide who would be responsible if something wrong happened with the drones. And basically the point was oh it would it will be the operator because he pulled the trigger and then you just skip the whole hierarchy behind that.
And I feel like like in some instance I it's it's the it's the goal that is looked for by the political leaders. try to dilute the responsibility and put it on the shoulders of the lower ranked people because just they pull the trigger, but we forget that when they pull the trigger is because they've been allowed to do so. They do not pull the trigger by themselves, right? And then you have lots of people that have made that possible at some point, right?
And we try to restrict that and to say, okay, you pull the trigger, then you're responsible. and plus the problem that we have with this system is that we try to define the responsibility a priori which is even before something wrong happens right which is crazy.
Zach: Yeah. Oh sorry responsibility a priori. So what does that mean exactly?
Emmanuel: Responsibility that usually let's say that you ar someone right you're driving your car and you're killing someone. you will be suited for that and then there will be a trial and then afterwards there will be an inquiry to see what your responsibility is. But you do that a posteriori after the fact happened right which is normal because then you can see okay what happened and then you can
Zach: Yeah after the action and
Emmanuel: Yeah and you can go backward and try to see what really happened and what who was responsible for what who made this kind of decision. But when you decide who will be responsible a priori it means that you will say okay if you the drone operators if something wrong happens that will be on your back. Yeah and that that's a big problem because I don't want that and then I don't want to be an operator at all because I feel like that that's not fair right there is something wrong because you cannot decide that I will be responsible before something happens.
Zach: Yeah, because you're as an operator, you're getting commands from higher ranking officers. You're doing target assessments, I assume, with teams of people. It's not just you who's sitting in a room with a controller.
Emmanuel: Usually in your cubicle, you also have a legal advisor that will tell you, okay, it's legal, it's not legal. So, there is a bunch of people that are responsible. even the people that actually program the drones, the provider of the drones, the people that there are lots of people that are involved and the responsibility must be shared. This is why usually you do the inquiry afterwards.
Then you can see okay who were the people that were involved in that and let's see who was responsible for what and then you will attribute your responsibilities and then you will decide upon the sanctions right but if you do that appro it means that will always be the same person that will pay for the you know the group that doesn't work so it's it's really unfair it doesn't work
Zach: Yeah it's and it seems like things are only getting more convoluted with companies like Palantir because now not only do you have humans doing target assessments and getting commands from their higher-ups. But now militaries are contracting out their decision-m to a company like Palantir who takes all this data and they create ontologies. And so they'll say who's a threat, who's safe, you know, all different types of classifications. And the military is starting to rely on these ontologies more and more.
So, if you're a drone operator, but you're taking a decision that was made for you by software from Palantir that says this person is, you know, 79% chance that this is a leader of a terrorist cell and it ends up, you know, being a guy on the way to a wedding for his daughter in Pakistan or something like who is responsible? Is it the drone operator? Is it the software engineer in Silicon Valley in California that's 6,000 miles away? Like how do you even decide?
Emmanuel: Yeah, that that's that's right. But that that's why you cannot make the decision ahead and there is a lot of discussion about who would be responsible if and I think that the question is not is not clearly asked. You have to okay let let's go let's try to make sure that you avoid problems. Let's try that during the development, deployment, the use of artificial intelligence or drones or whatever. You make sure that every single stakeholder is aware of his responsibilities and just try to make sure that all responsibilities are clear are clear for everyone.
Then if something happened then you will be able to say okay you did that you decided that for example political decision maker might be decide might might decide to use drones why they would have been also possible to decide using something else. So the only fact that you decide to use drones make you responsible for what happened even if the operators is pulling the trigger at the very end of the day right. So I feel like sometimes we are we are trying to make things easier but it's it's always to avoid putting the responsibilities on the shoulder of the people that are really responsible.
Zach: Yeah. I just had a dark thought, which is that in order to train these AI weapons and AI algorithms that are now deciding who's a threat and who's not, you need conflict or, you know, you can't, if you're going to train a weapon, you need conflict to gather data from an actual war zone in order to determine how good the decision-making is and like to tweak it and stuff like that.
Will we see politicians and not and not just politicians but also tech CEOs sort of like lobbying to go to war that won't affect them in order to not just make money but to train these weapon systems and algorithms that they're trying to sell to the military. It's like like almost like conflict breeds conflict and it's just like a cycle of like AI weapon systems conflict and it like just keeps making it
Emmanuel: I don't know. It's it's really hard where to know where it's it starts exactly? Is it from the civilian world that are that is pushing towards sometimes I can I can feel like for example in France even the tech industry is pushing towards you know military operation because you know they have a market they have to sell their product and they there is big money behind that.
So you feel like there is a lobby but it's what you know again it's it's a mixed responsibilities between the politicians that are that are making decision upon what have been what they've been told by by tech leaders but also tech leaders that are that have been told by the politicians or even the military leaders that for the sake of the defense of the nation they have to contribute to you see that it's it's kind of a specific environment that is feeding itself with a perspective which is okay. we have to make money. If we want to make money, we have to fight wars. If we want to fight wars, we have to do this, do that.
And that's that's the kind this kind of you know, ecosystem that is created around money and the defense of the nation. And we all know that for example, that's I think that's the case in France and Europe today. you know, the financial situation of France. And we I guess that we see in the rearmament of France, a way maybe to get a bit money a bit more money and to escape this kind of situation.
So you know it's it's and especially I know in the US and France you have a really strong interconnection between the civilian world tech tech CEOs and the and the military and the political you know political leaders and so they're all interconnected and they all see their interest. So if the interests of political leaders in terms of you know let's say domination or even a question of reputation on the international scene just match with the expectation of tech leaders about okay I will make money out of that then they will go to war altogether. Yeah.
And they will do it with no you know concern with even no ethical concern I would say.
Zach: Yeah. It's been an interesting shift to watch in the US because there used to be a at least when I was growing up a deep divide politically between Silicon Valley and the government, especially the Department of Defense, any sort of regulation. you know, tech CEOs thought government working with the government was corny, out of touch. They didn't want to be seen as being hand in hand with government operators. And now there's been a total switch in the last 5 to 10 years, which is now for a lot of these tech CEOs, the government is their number one customer.
And they're embracing working with the government and being part of the regulatory process between tech and government operators. And it's just it's just been a total shift. And I know it won't be exactly the same in France. There are a lot of differences obviously between the two countries, but I'm I'm curious if there's been a similar shift in France as well.
Emmanuel: I don't really see a shift here. I feel like how can I say that? I feel like, you know, it's it's always the same thing. I feel like there is a difference between what people are saying, let's say tech leaders or techs are saying regarding the relation to the government and the reality of the relation.
It's exactly like us with China with always saying, "Okay, China the bad guys." is no but we are know we're doing business with them right so there is the communication which is okay we know that you don't like Chinese people so we'll weured you about the fact that we are not dealing with them but at the same time we are dealing with them and I think in the US or in France exactly the same thing right the military industry or the defense industry is tightly linked to the government whatever the government but on the on the at the public level when they have to communicate especially you see that for example Example, we all know that in the military and the tech industry, people are not really keen off democrats thinking, right?
They feel like it's it's killing their innovation that they are against lots of things. So, usually when you have a democrat government, political leader would not be really happy to see that they are working working hand in hand with this kind of government. But now that we have that you have basically Mr.
Trump you see that there is because once again there is kind of a convergence of point of view of interest and I in France we don't really have this kind of shift because basically I don't feel like whatever that might be leftist or rightist government I feel like the military industry is still there so they they're not really happy with decision that are made by the left part of the government but anyways they don't have this kind of problem so we don't have this kind of shift there think our connection is really strong here.
Zach: Yeah. No, that that's a good point. There's the actual relationship that the government has with tech and the government has with other governments like US and China and then there's the verbal exchanges. There's the communication to the public on the surface. And you're making me think about are you familiar with pro wrestling like WWE, WWF, like John Cena, John Cena, The Rock, where they come out and they talk a lot of [profanity censored in source] to each other and there's good guys, there's villains, and then behind the scenes, a lot of those guys are friends.
It makes me think about how a lot of politics is pro wrestling where on the surface, you know, in front of the public, you know, maybe at a press conference, they'll go at each other to sort of posture for their own country or for their own company. And then behind the scenes, they'll be like, "Oh, yeah, let's, you know, I was just talking [profanity censored in source] Let's grab a beer."
Emmanuel: No, that I think that's exactly the analogy is perfect. That's exactly that. But once again, not something new, right? The once again there is there are two kind of ethics. You have the ethics that the real politics and you have the ethics of the you know the constituency and the basic population right and you have to always find a balance between two things right so you have to even pretend things that you are not to doing in the real-world.
So that's one once again circling back when you are listening to a political leader speech or even a tech leader speech you have to take it for what it is it's a communication exercise that's it right it's not analysis it will not provide you with clear information about the very nature of the relation that those people have with each other so you have to dig a little bit behind that but I really like the analogy was pro that exactly that
Zach: Yeah no while you're talking about it just popped into my I'm I'm curious before we moved on to what you're currently doing now with ethics applied to AI, I wanted to ask you if you could go back to that period when you were working on drones in the military, what ethical warning do you wish had been taken more seriously?
Emmanuel: Responsibility. Back to responsibility. I think it's something that has not been taken seriously and it's still not really taken seriously. and I would say human responsibility. We we've we've slowly slipped toward less and less responsibility and moving responsibility to the even the technical object itself. Not even the operator or someone the human being. Now we're just trying to put the responsibility on the shoulder of the technical object. Right? You know I think that people are not are still not aware of our responsibility.
I mean not only leaders but also our own as individual responsibility what's what is happening right because when for example I'm I'm slowly accepting to be a culture to the use of drones the use of AI the use of little weapons I am part of the problem right I have a responsibility but it's easier to say oh it's because of the big techs the gaffam or the bad in China or is the political leader this kind of thing and I think that responsibility has not has not and still not enough you know taught to individuals.
Zach: Yeah, absolutely. So you slowly moved from the military side of ethics to the AI side. What compelled you to make that switch? What drew you towards the ethics applied to AI?
Emmanuel: It's it's basically the fact that once we had the drones then slowly popped out the fact that we might have some level of autonomies. So that was the MIT were talking about that.
So we were not talking about AI but slowly AI entered into you know the ecosystem used to make systems more and more autonomous and then we discovered slowly the use of artificial intelligence and then it has become something really big with GPT and these kind of things out there and slowly we had to move to artificial intelligence because that was kind of the big focus of most stakeholders right so but the move from the drones to AI was basically the level of automation that were that were used in military.
Zach: You've yeah you've written a ton about the western dominance in AI development, the lack of multicultural input. Can you talk more about that and what makes it such a problem?
Emmanuel: I think it it's a problem to me, not for not to most people. I see BA basically I feel like ethics is really a worthentric thing right when you go to China for example there is no suching ethical thinking that's not the way it works it's really hierarchical so when you have a legitimate authority that is making a decision it will provide you with orders and you will have to obey based on the fact it's a Confucian perspective which is because you're the authority because you're legitimate I trust that whatever the decision you will make it will for the benefit of us.
You know, that's the way it works with your parents, with the president, with whatever the authority. so they don't have this kind of ethical thinking. If you go to the Muslim world and basically Muslim world is a really vast and really heterogenous world, but if you go to country like I don't know iron or you go to Saudi Arabia, it's basically the same hierarchical perspective where ethics has no room at all.
Right? everything is aligned to with religion and with the prophet the hadit what's the words of the prophet this kind of thing so you don't have as an individual to think about ethics in the western world we because of our move toward more and more individualism ethics has been splitted into small parts right so each of us is now entitled to speak about his or her own ethics So you can't put yourself just step back and look at the rest of the world and say oh my stance is different. Right?
So we have a specific perspective on ethics that we are now trying to spread to the world because we feel like we hold a truth about what is acceptable what is not acceptable because we are no developed countries because we have we are the heir of the enlightenment because we have a strong background in philosophy and this kind of thing or because you are the US and you feel like you are an empire and you have a specific you know duty to spread your world to the world.
So it it's this kind of western perspective that what we have what we horn in terms of philosophy is worth being spread without considering different perspective right and we feel like that's that's right that's normal and we and with AI we've done exactly the same for example concerned about discrimination between genders which is definitely a problem in the US that might be a problem in France or elsewhere is not a problem when I go working with Saudi Arabia right because it's culturally root in the fact that in religion basically that actually you discriminate between male and female, right? but we feel like even I've seen French people or European people or even US people working in Saudi Arabia and trying to lecture the Saudis about the fact that they should not discriminate and you feel like that doesn't work because if the reverse would happen, let's say Saudi people would come to the US and say now you have to discriminate, you would not accept it, right?
That would be like okay that's that's weird because it's not my culture. So it's always weird to see that we feel like we hold a truth that our values are universal and that we are entitled to spread them and even to impose them to the rest of the world. Back to the military I've experienced that with democracy when we went to Afghanistan say okay we have to spread democracy based on the fact that democracy will not fight each other which is okay kind of theoretical perspective and then we try to establish democracy in a country that was not ready to accept democracy.
Zach: Yeah. Right. And you see it worked out so well for both people involved. Back to square one after 20 years there. I think it took I think it took the Taliban about six minutes to take over again once the US
Emmanuel: Yes. Yeah. Yeah. Yeah. Actually, that's crazy.
So the point with this cultural dimension is that I feel like we lack we lack let's say critical thinking about our own knowledge, our own values. what what's what we think and plus I feel like it's it's a bit dangerous because if you really look I used to say that it's it's quantitative so it's it's not the truth per se but the western world represents something like 7.5 or 8% of the overall population which is nothing right so it's it's funny for a country like France or the US that is promoting democracy to say okay us the minority we are lecturing the rest of the world about what is acceptable what is not
Zach: Yeah I mean and that's all that's something too that I've realized as I've traveled more outside the US and I've gotten to experience different cultures is that this hyperindividualistic outlook is unique to where I'm from, but it's not necessarily part of where other people come from around the world.
So to me, I'm, you know, I'm basically sold this idea that if you're not looking out for yourself as an individual and trying to stand out that you're going to somehow have a less fulfilling life or have a, you know, let less will come your way. you'll get less opportunities and that leads to things like you know as soon as you graduate college you want to distance yourself from your family and prove that you can exist on your own and you want to move far away and you want to start your own thing and then you go to other countries you know it makes me think about Japan where there's there's such a higher I know at the moment like there's less and less people having kids in Japan but there does seem like there's much more of a communal aspect to families taking care of each other.
And you don't really want to stand out in Japan. Like it's it's such a it's such a and I don't mean this as a negative. It's much more of a submissive culture where you kind of you don't you want to blend in and if standing out is seen as you know blasphemous in a lot of ways. You don't like you don't want to you know make a scene. and you don't want to make noise where in the US it's like no make noise like shout you know complain to the manager you know be on your phone on the subway like all all these things of like no it's about me it's about me it's about me and the rest of the world does not share that
Emmanuel: I think you know there is a big cultural difference that I don't know if you've heard about the MIT Moral Machine experiment which is available online and if you haven't heard about that I the MIT Moral Machine experiment MIT MIT Moral Machine MIT MIT Moral Machine experiment experiment that that's really funny thing. It's about obviously autonomous machines, right? And what they've what they've discovered which is basically something that is normal is that there is a big difference between two things in the world.
You have people that are really collectivist in the way they see the world and people that are really individualistic, right? And I think that when you're talking about Japan, it's collectivism is key. It's not because they are submissive. It's because they feel like the interest of the nation is more important than my own interests.
Zach: Right. Yeah. Yeah. Collective collective is a much better word.
Emmanuel: Yeah. Exactly. And I feel like in France or in the US the same, right? We we're much more turned toward our individuality and we don't care that much about the collective interest, right? And even the political leaders, we can see that at the same time. But when you go to I would say Japan when you go to China that's exactly the same. If you go to some countries many countries in on the African continent you will have the same perspective about collectivism. if you go to northern America you have collectivism.
Japan is a really specific case because it's an island and if you want to survive on an island you need a collective perspective. You need to defend yourself against the rest of the world. But it's really specific. It's cultural. It's geographical. It's historical as well.
So and in China it's it's much more about the Confucian perspective. so you got lots of things that make some people some human communities more turn toward collectivism than to individualism right and that's the big difference and that's something that we cannot understand and because you were using the word submissive or because we see for example China as a dictatorship
Zach: That that's my American bias. I see collectivism as submissive.
Emmanuel: Yeah. Yeah, but that that's totally normal once again and you accept the fact that you're biased, which which is great, right? Most people they do not even accept that they're biased. But the point is not to say that I agree with the situation, but if I look at it in a really objective way, I'm just labeling things in a way that is biased because my perspective of collectivism is dictatorship. It's it's you know the analation of the individuality, this kind of thing.
While actually when you put yourself in the shoes of Chinese people when I when I myself go to China when I'm teaching in China in Shenzhen for example what I can see is that actually people they're really happy with that they know that they can't they can count on each other right and they feel like they are a collective entity. Yeah. in which each individual has its responsibility. It's something that talks to me because I was in the military and the military works on that exact same principle. Right. the collective is much more important than the individual in the military. Yeah. If not, it doesn't work.
You will have problems, right? And for the survival of the nation in China, in Japan or if you go in some places in Africa or southern America, that will be exactly the same. But to us, once again, it sounds like really pedrative and we feel like we're not really comfortable with this idea.
Zach: Yeah. No. And I think travel is a big part of being accustomed to that and living for not just visiting but living other places long term. I've I've gotten the opportunity to stay places for three months at a time working remote close to a year working remotely which has been a big gift since co being able to travel and kind of like live in a town and just see how things work. I wanted to go back to the US involvement in the Middle East because you mentioned the example of Afghanistan and I was thinking that like when a when a country like US becomes involved in Afghanistan, it's very it shakes that part of the world.
It's very obvious what's going on. You could see soldiers on the ground. You see videos of people dying, people, you know, getting blown up. You have all this like very visceral evidence of what's happening now. If you think about AI and western dominance of AI, I don't know if you see it like this, but when I was reading your work, it kind of seems like a more subtle invasion in a way where it's like it's not super obvious how it's going to change the rest of the world.
But when you have you know 70% I think you said it was of the AI ethics publications coming from the west where not even close to that percentage of the population what you said it's like seven or 8% of the world is western like do you see it as a this a sort of subtle invas invasion of like thoughts and idea shaping through western AI dominance.
Emmanuel: I think it is I don't think that there is I think that I don't think that is intentional. I don't think that there is kind of a strategy making it subl and you know this kind of thing. But I think that the impact is clearly that and it's not only me. I had the chance to work for example with the Indian government at some point and they were really concerned about the fact that western values were spread upon the country. that actually some of those values were not really aligned with Indian values or that might even threaten Indian values.
So they were like okay we have to think about it right and we have to work on that like of an ethical perspective that might be a native ethical perspective based on brahmanism this kind of thing and you can see differences there I've seen that working with [organization unclear] which is the equivalent for the UNESCO for the Islamic world right I'm I'm now a member of the chair and I worked on the charter for AI ethics with [organization unclear] they had exactly the same concern about the fact that most of the AI ethics narrative has been set by the European Union and the US the western world at large and the world like sometimes it's not really it does not fit our expectation is does not fit our values does not our interest so there are two things it's like globalization first you have kind of homogenization of perspective that is really subtle but at the same time you have clusters of people that are actually fighting against that right so you have also this kind of reluctance or rejection of what's coming from the western world and I can see some clusters in India now in China because China has recently said that they would they would become a leader in norms that was not exactly the point until now.
So they've decided that they would have kind of a leadership in norms in AI. I can see that once again from the let's say Arabic peninsula or the what we call the Mina region you know Middle East and Northern Africa.
So I can I can see here and there people that are that are now considering the fact that maybe this subtle you know influence that has been made possible through AI is becoming slowly a problem right so they're just trying to say okay now we have to build something that is really aligned with our interests and with our values that might be values coming from religion that can that might be values coming from you know tradition or history, whatever, but they're just trying to anchor the reflection into their own specific settings.
Zach: So would that be like let's say we have a ChatGPT which comes out of US tech companies would that be like you know India or Africa having their own form of that like some something that doesn't necessarily pull on Western influences? So I'm trying I'm trying to think like how that would work in practice. Like what would that look like for a for different sets of people around the globe to remove themselves from the western ethical dominance of AI?
Emmanuel: I think yes you most countries then now they they're looking for you know sovereign LLMs. they're trying to develop their own generative AI system, not being too dependent upon whatever you know western companies but at the same time it's really difficult because you need resources to do that right you need to have money you need to have the people that are able to do so you have you need to have the political will to do so it's really difficult so India is trying to develop its own system they're trying just to at least the first step would be and what they're doing is to decide what are our values what are and what are the values on which we can build something and then we'll see for the technological tools etc because you know that AI is not only generative AI that might be lots of other systems right so they're just trying to see what is the moral framework and then they will decide upon what's inside this moral framework and what are the values on which it is built and then you can you can implement that in your own system or you can even take systems from the shelves for example you are you using GPT or Gemini or whatever right you can have contractual closes that would say okay for our specific use we will have specific rules right so you can adjust and or you can ask for adjustment that happens for copilot these kind of things so you can definitely do that too so for countries that have I would say that are that have the money and the resources is to do so.
They will develop their own LLMs, their own systems. For other countries, they might also think about having their own ethical charter, their own ethical values and built on that and ask providers to adjust their product to their specific need and values.
Zach: In your experience, what do you think are the biggest blind spots of Western ethics applied to AI?
Emmanuel: I think that the big problem that we have is principalism which is we think that we can regulate things morally speaking or ethically speaking through principles and when we've set principles say okay job done and then we do what we call compliance when you're a company you will check will just check that you are that you are aligned with the regulation right you think boxes okay am I transparent yes no am I am I respecting non-discrimination Yes. No. These kind of things, right? So, we rule the world through principalism through principles. Principles and I think that we feel like it's it's the right way.
But when you do that, basically what you are setting is principle that are really high level of abstraction but that will not be really contextual and then you have a big problem in the operationalization of those principle and especially when you move from one culture to another. So I think our blind spot is principalism which is itself based on this perspective that there are universal values which is totally untrue.
There is no such thing as universal values but we are so confident in that we believe so strongly that there are values that like democracy that is universal that you know equity or equality that are that it's not true but we cannot understand that it's not true even if fact actually it can be proved that it's not true people would say no no it's universal and then because it's universal then we can have principles that are universal that's basically the way we think which is totally biased.
Zach: Yeah. I because the world is so global now with connections, we can see more clearly how people live in other cultures. So, it's becoming less and less of an excuse to say that you're oblivious to the different values of other cultures. So I'm wondering what still makes the West so confident that its moral assumptions especially with AI should apply everywhere
Emmanuel: Because I think that I will I will tell you about my own experience. Right. Yeah. When we decided with my wife and our son to move to Canada, we felt like okay Canada is kind of like France, right? If you Quebec and there is this kind of cultural proximity. So you have a perception of what Canada is from the French side. Once you arrive in Canada, when you land in Canada and you start living with Canadian people, then you start discovering that actually you did not know about the culture. You did not know about Canada. You did not know about what Canada is exactly. And then you start seeing the difference.
You start start discovering this kind of difference. And I think that it's not because we have access to the rest of the world. It's not because we have you know TV entertainment that has that are showing how beautiful the US is or how scary the China is that we have a clear understanding of what it is. So we have perceptions about the world but we don't have knowledge about the world. So perceptions usually they will not make us change our you know stance on things.
So we feel like we know about other cultures but we are really rooted in what we know what we master which is our own beliefs that are that are deeply rooted right and then when you say that there are universal values in France even when I myself teach and I say okay there's no such thing as universal values even in classes where I have for example students coming from different countries for western people and even for Chinese people. That's that's true as well, right? It's hard for them to believe that even if they themselves live in a cultural, you know, mixed environment, they they
Zach: You're saying it's hard for other people to believe in the universal values of the West or
Emmanuel: Yes. Yes. I think that even China, they don't think that they have universal values. They don't think that way, right? M but what I mean is that you will still believe that there are universal values unless you experience something that shows you clearly that there is no such thing as universal values.
If you just you know experience the world through screens or through books or through YouTube video or whatever then you feel like you know why you don't know but your mindset will not change profoundly right you will still believe that there are universal values and even sometimes you will reject things that you see because they are not really aligned with what you think so you will adjust what you experience to what you feel but when you live in a situation and if you've experienced that moving in another country or living in another country even for 3 months then you feel like okay now I'm really experiencing the thing like in the military and what do I do with that right it's it's not it's not it's no longer about what I think it's about what I leave for real and experiencing things for real is different than experiencing things through through the digital
Zach: Yeah it's it's funny it does feel like such an American centric thing to want the rest of the world to be like you and then you don't understand quite why It's not It's like hard to put yourself in the mindset of other people. And I I've been guilty with this. I I' I've been guilty of that.
And I'll say that I've definitely gotten better with traveling to I've been lucky enough to travel to many different countries, but like I even, you know, I would go somewhere and then I'm like, "Oh, you know, like I'm in I'm in like a remote village in Panama and I'm thinking the service is slow here and you know, why can't I get pizza or you know, like this, you know, the like things about crossing the street like why like no one crosses the street here illegally, but in New York you do that all the time. Like jaywalking is just a part of life.
Like I noticed that in the Netherlands like people like really obey traffic laws and I'm like it's a red light but I'm looking and I cannot see a single car for miles down the road. I'm like why won't these people just cross the street? And I did and I and I was like I guess that's just the culture here. People have respect for traffic laws. which probably is good for you know accidents and bikers getting hit and stuff like that. I'm sure it, you know, it pays dividends. But to me, I'm like, why won't these pe like this is this is crazy.
And then like over time, you sort of like get outside yourself, get outside of your own ego and realize that, you know, every single person you're passing grew up totally different than you and they have a completely different set of norms and values. and you're sort of projecting your own experience and views onto them, which is not the correct thing to do.
Emmanuel: But the point with what you're saying is actually when you experience something, you start questioning yourself. You start questioning your own practices. You start first you ask why why are they acting the way they're acting? Right? That the first things that pop up to your mind and then you say why do I why actually how can I say that? Why do I act in a different way? Then you start questioning yourself. And there you can start questioning your knowledge, your beliefs. Then you might start slowly changing your mind and then you might even conform to the new rules, right? Oh, that that's not that stupid.
At the very end of the day, for example, it will lower the risk of accident. That's great. So, let's do it, right? it's not a problem. And then you will change your mind.
But once again if you when you experience the world from afar when you and especially when you experience the world for example in France through the media which are basically all biased and not intentionally right it's not there is no bad intention behind that but all the people that are making you know comment about what's going on in the US in China in Saudi Arabia or here and there are bias in the way they're doing so if you watch the media the news and you always watch the same news you will always be fed with the same biases Right.
So you feel like you experience the world but you are not experiencing anything actually.
Zach: You're getting it. You're getting news through a bias kaleidoscope of whichever view you're choosing to look into.
Emmanuel: Yeah. Exactly. This is where once again critical thinking is key if you really want to go to dive into ethics and to go beyond just what you've been sought.
Zach: Yeah. you you've written a lot about ethics applied to AI when it comes to African countries and I wanted to get into that a little bit and you've said that saying that Africa must be included quote unquote in ethics applied to AI actually reinforces inequality c can you expand on that?
Emmanuel: I feel like it's it's once again if I make an analogy personal analogy when I when I went to Canada you know Canada is a really welcoming country is really promoting diversity welcoming lots of people from around the world. so they are actually welcoming people but they say that diversity is important but they say that diversity is important as long as it fit in your cultural you know mindset. So meaning that people must enter in your bubble. You will not enter into them or you will not try to see the interconnection between their world and your world. You want them to enter in your world.
Meaning that actually at the very end of the day, you don't really care about their world. So when you say that we will include people, it means that you're taking them like you just, you know, you're the big guy and you want them to enter into your room and to adopt your rules. The point is not to include anyone. is to work and in end with others, right? Or even to find I usually use the this idea of the bridge, right? You have to meet in the middle of the bridge. I can tell you, okay, look on my side of the pond in France is better than in the US.
So come to France and you will tell me, oh no, in the US it's better than in France. So you we can have this kind of discussion. But what is interesting is just to meet in between and to look at each at each side from the middle of the bridge and then this is where we can we can actually meet and discuss honestly and we can have this kind of real collaboration. If you come to my place, you will have to adjust to my place. If I come to yours, I will have to adjust to your place. So including means that I am superior and you will just come into my you know moral setting or my moral environment.
Zach: It's like we're allowing you to enter the world.
Emmanuel: Exactly. Yeah.
Zach: Yeah, I was I was just going to say if a country like you the US, let's say we include Africa into the development of AI LLM, it seems like you wouldn't really be including the average African perspective because you'd probably meet with political leaders in Africa, diplomats, people that are much more westernized than the actual African person.
So by including Africa, that's probably just like, oh, we're going to meet with, you know, higher up tech leaders in Africa or people that are political leaders that don't actually represent n an African perspective like if Africa had their own LLMs and data and algorithmic autonomy.
Emmanuel: That that's true. It's the it's it's one of the point that have discussed with the UNESCO for example years ago which is the fact that they were really biased because everything once again is really western centric and all the diplomats that were working at the UNESCO even if they were coming from the African continent from India from southern America they were really westernized as you were saying because they've been highly influenced in their practices maybe in their studies maybe some of them have been raised in the western world and they did not really represent the interest of their home countries. That's for sure.
Right. So when we say that we're including those people, I can see that if you go to Canada and you if you look at people that are from Africa that are working in AI, they're not working in Africa. Basically, they're working in research center in the US, in the UK, in Canada, in France, in the Netherlands. That's that's the point, right?
And so those people I would not say that they do not represent at all their culture but they do not represent it correctly because once again they are they are influenced because also if they want to exist in this ecosystem the western ecosystem they also have to adopt the practices and the thought of the western world right because if you do not do so then you're ostracized and no one will listen to you right so if you want to be heard and if you want to exist at some point you have to conform to the rules So you're no longer an African guy. You're no longer an Indian guy. You're no longer a Brazilian person, right?
You are now part of the ecosystem that is telling you what you should say, what you should think, the way you should work, and the way you should understand the world. Even people that are promoting, for example, the Ubuntu perspective of Africa, they're they're just, you know, paying lip service because they're not really diving into it because they know that if they dive too deep into that, they might have problems on their own, right? Because the carrier would have been impacted by that.
Zach: What is Ubuntu exactly?
Emmanuel: It's anonymist perspective that is that is that is current in Africa.
Animism is basically this idea that everything even material things are inhabited by a soul or gods these kind of things right saying that for example you have to respect the same way a chair maintain as a human being or an animal right so they do not make animism does does does not make differences between things like we do saying that for example human beings at the at the top of the hierarchy and then we just have to control the rest of the world that is that is here just to please our needs. animism and Ubuntu is one of this kind of wisdom that you can find in Africa which is based on animism.
Zach: So the these would be Ubuntu and animism would be examples of wisdoms that might shape an African AI system that wouldn't wouldn't be present in the west and it wouldn't be shared by the west.
Emmanuel: So no yeah I mean you can find out also sorry you can find out also in Shintoism in Japan right it's animism in Shintoism when you have an object it is inhabited what we call the gods that are called kami and then when you have an object you do not destroy it you not put in the garbage at the very end of the day you will have kind of a you know a process to make sure that the camis are leaving the object in the right way and you will not destroy and you will not break things just for this the you have the relation to technology or to artifacts or to objects is totally different.
You have respect for things even material things right due to the fact that they're inhabited by gods or what I saw. So yes, it's something that might definitely be important and that might be used in the way we think the world, the way we think our relation to technological tool. You know that in France or in the US there are questions about should robots and AI have rights. That's the question for example and this is exactly the question that might be answered through the animist perspective.
Zach: Yeah, you could see the difference in arguments where if you're giving respect to an object based on a spirit that inhabits a chair or a picture, you would say that well a robot deserves respect, too.
Emmanuel: Exactly. Yeah.
Zach: Not even on the basis of consciousness, but just because it's also a thing. It's a material inhabited by a spirit. Whereas in the west, you know, it gets it's a totally different argument.
Emmanuel: Exactly. that that's why the question is so pregnant here is in Europe and the western world is that we're wondering if we should give rights to those tools just because to us technology is a tool it's a it's a mere tool and when we are we no longer need it we just can get rid of it right we can put it in the garbage that's it and once again in the enemy's perspective it's totally different so that's something I don't I don't mean that enemies perspective might find or provide us with solutions, but it might enlight the discussion.
Zach: Yeah. Yeah. It's I've actually been thinking about that recently because, you know, inter interacting with a tool like ChatGPT, it can start to feel like there's, you know, going off of the animist perspective, like there's a spirit that's somewhere deep inside there talking back to you. So, the other day I said something where I was like, you know, find me the, you know, top 10 resources for this subject on AI warfare, let's say. And I said, please. And I was like, why did I say please? Like, I'm not like I'm being polite to an LLM.
But on the other but on the other hand, I was like, wouldn't it be nice though to like cultivate a polite relationship with AI? Like I like even even if there's nothing there like it never turns into anything that we can understand as conscious like would the world be in a better place if we treated systems that have the appearance of human intelligence with some sort of reverence or respect and we're not just like treating it as you know input output like give me this now I'm telling you to do this do it now I don't know it was just a weird thought
Emmanuel: I No reason I think we I asked myself exactly the same question at some point saying okay why why why should I say please f first the point is that it doesn't hurt why it doesn't change my life to say please I'm used to say please in my everyday life so I can say please to an LLM not that's not problem the second thing is that you know that technically speaking when you say please the system will learn that you're polite and will interact with you in a more polite way right so there is kind of a relation that you're creating through the wording that you're using when you're prompting a system like that.
But I feel like being respectful to objects, to animals, to nature to me is not is not a big issue. Not at all a big issue.
The point and once again it's back to individualism and collectivism. the fear that we have is that at some point this system might maybe geopodize our individuality, right? and that's the fear we have like enemies they don't have this kind of perspective they do not fear that right they have kind of a an interaction they feel like they're part of a big network in which you have material object immaterial object that you have animals that you have plants that you have human beings it's kind of a big network and everyone counts on the other to survive in the system while us we feel like okay we are kind of semi- good on the top of the hierarchy right and everything is just here to please us.
So we are using things like tools. You have people that have exactly the same relation that we have with technology as tools with animals, right? And that's exactly the same question. You get people that do not respect animals, right? And so you feel like once again I don't know if there is any kind of truth or something that is right or something that is wrong. But I feel like this kind of reflection might definitely enlight our relation to technology at some point. Maybe we can reject it and say, "Okay, we don't have to say it please. After all, it's just algorithms. That's for sure.
Or we can say like I say saying please doesn't cost me anything. So if I can say it, let's say it and I'm used to it because it's part of my culture as well.
Zach: Yeah. You when you were talking about animism, that was that was also making me think about all the other perspectives and the wisdoms around the world that don't get incorporated. Like you mentioned the Confucian perspective, also bu Shinto Buddhism.
I think there's like 300 million Buddhists on the planet and they have a very different perspective when it comes to the self than the West. that seems like they have much more of a like a less ownership centered version of the self which also applies to things like data like collecting data for software AI you know what does that mean for Buddhist perspective on AI versus western perspective where it's like taking data is such a deliberate offense of privacy where for you know Buddhists that means something that could mean something totally different if you have a different view of the self and what you actually own.
Emmanuel: Yeah, it's there are really interesting work that has been done by a Thai philosopher called Sorajang Ladder and he worked on the Buddhist perspective on ethics of AI and what is stressing is basically that the grounds for what we call for example respect of privacy is deeply rooted into this idea of you know ownership of myself my body my soul my mind my spirit this kind of thing and what Everything that is related to this self I own it. So if you want to use part of it, let's say data regarding that data that is related to me, you have to get my consent. Right?
But back to the enemies because when you talk about Buddhism, it's a Brahmanic perspective and Brahmanic perspective is animist as well. Right? The Brahman basically is the big spirit that is everywhere and when you were born actually your body just absorbed part of the Brahman, what we call the atman. But you do not own that. When you will die, it will go back to your atman will go back to the Brahman. So you do not own the atman. You do not own your soul. So you do not own the data that are related to things that are not directly related to your body, material body, right?
So you don't you don't have this kind of ownership about data and you don't have the same perspective about what privacy is. So this is the kind of thing that I think that once again might enlight the discussion for sure. One country that is really ahead of all others is New Zealand because they've taken into account the Māori perspective which is animist in some in some respect. It's called MA Māori perspective. Yeah. They have Māori perspective. Yes. Yes. Yes. And the Māori are participating actively participating to the elaboration of the rules and the recommendation and the regulation in New Zealand.
And now even the equivalent of the GDPR in France the regulation for data has taken into account this Māori perspective. So there is a strong work that is done to make sure that we are using that and that we are aligning with the Maui and that's something that is for example missing in Canada with first nation or in the US with first nation as well. I don't see that in Australia with Aboriginal people as well. and I feel like they they're they're learning a lot. So first the point is that they're learning about each culture which is great and then they're enriching themselves with other cultures perspective.
So it's it's really something that is really great when you look at the New Zealand work on data and ethics.
Zach: Yeah, the Māori perspective I was just looking up the concept just to clarify. So yeah, it says the Māori concept in New Zealand is the life principle, vital essence or spiritual force within all beings, objects, and ecosystems. also used by the North African Berber people known as the Moors. So yeah, that's that's really interesting that it was it it's able to make its way from North Africa all the way to New Zealand.
Emmanuel: But you know the relation is once again animism right the spirit and especially in the Māori perspective or even other perspective in the world sometimes for example mountains or trees or rivers they're inhabited by the spirits of the elders the people that were you know your ancestors that actually died and moved back to the earth and then you see that everything around you is inhabited by the spirit of your ancestors. So you have to respect those things, right? I've seen that when I was living in Australia. in the Aboriginal perspective, it's exactly that, right?
So there is this perspective that is really enriching to us. Whatever we think about it, maybe we can we will reject it or we will adopt it, but at least it helps us think about our own conviction, about the relevance and the legitimacy of our own beliefs. And maybe it might also help us to make better decision when it comes to AI things. And plus plus back to what we were saying earlier, it will help us avoid the this kind of cultural tyranny from the western world.
Zach: Yeah. Le let's let's take a quick break. I just have to go to the bathroom real quick. So something I wanted to get into. you wrote a fascinating paper called anticipating the geopolitical impact of advanced AI and there's an excerpt that I wanted to pose to you for your thoughts and you wrote that quote generations who grow up in a world full of AI generated and mediated content will develop new relationships with technology and information.
Some will elevate AI systems to the status of technical demi- urges with transhumanist movements reinforcing the weight of AI systems by encouraging the augmentation of humans through these technologies. Civilians and soldiers alike will be increasingly connected via exogenous equipment such as augmented reality goggles or exoskeletons and devices implanted directly in or on bodies. which sounds like total Black Mirror apocalyptic situation.
So like I wanted to ask you like are do you think we'll be living in a world where war and everyday life start to blur if civilians and soldiers have access to this type of technology where we're like constantly evaluating threats, you know?
Emmanuel: Yeah, I think I think that there will be this kind of situation that will be very difficult that because everything will be blurred very very blurred. I feel like you know the first the difference between reality and what is you know something that has been artificially created will be more and more difficult to differentiate and that will be true for the military that will be for the for the civilians.
I feel like we are slowly entering in this world where everything is possible and toward a world that is totally totally I mean foreign to me because I'm old and I'm happy that I will certainly not see this kind of world but that will be the new normal in the future.
So yes everything is really blurred and it even even the difference between the military and the civilian is already blurred you know with you knowi civil contractors and this kind of thing but it will be even worse in the future because even war per se I would say not war in the legal sense of the of the of the word but war at large is no longer a matter of military people right? war can be waged also by cyber criminals. It can be waged by teenagers on the other side of the planet and threatening you. Right?
So I think that and it's something that is not that is not new but it has been multiplied or demultiplied by artificial intelligence because we know that for example the difference between security and defense is really difficult to make. We know that now you have civilian actors that are acting in wars. you have military people that working in the civilian world. So everything is kind of mixed up. But with AI, what makes things really different is that first it will touch many more people than what happened until now, right?
Because lots of people can enter into this world and because you will you will digitalize the world and once again you will create kind of a competing reality and in this competing reality you can become a military guy you can wage a war you can kill people you can harm people or military people can you know become you know you know peace promoters I don't know so everything becomes possible and the way that we are actually in the US or in France and Europe we are functioning which is really a functionalist way of thinking the world you have your own function as a military you have the function as a political leader you have a function as I don't know someone that is working in the else industry will totally disappear and everyone can be where he or she wants and that make things really blow differently
Zach: Do you see AI augmentation and the creation of these different realities ities. Do you see it creating this like a new global class of inequalities like between enhanced and non-enhanced populations?
Emmanuel: Maybe that's that that's definitely a good question. I would say yes because there is no reason that it would not happen at some point. You will have ends people not ends people. You will have people that are mastering AI, people that are not mastering AI. So the world and that's that's one of the big thing about biases and about inequality and people that are trying to fight against inequality is that whatever we think about it. I will tell you something that might be a little bit weird but the beauty of the world is based on inequalities. If we were all equals that will be really boring and that would not last very long.
That's not possible. That's not possible. Right? So obviously there are people that are suffering from inequalities and that's really sad. there are people that are you know benefiting from it that's great for them but we are living in a world that is inequal nature and I back to you know philosophy and what would call the state of nature with Hobbes you know man is a wolf to other men the point is that when we are interacting with others we're just trying to you know to get as much resources as we can so it's it's always detrimental to someone else.
So when it comes for example to augmentation if we feel like being augmented will give me some assets that other doesn't have I will go there and then I will create this kind of inequalities I will create this gap between people that have that have the means to for example get augmented or people that do not have the means to get augmented right so yes obviously I would not talk about classes because it brings us back to Marxism this kind of thing and I feel like it's kind of misleading but yes you will have inequalities but the world is made of inequalities and that might be sad, that might be great. I don't I don't know.
I don't know if it's right or wrong. Factually speaking, it will not change, but it will be increased with AI, with new things like new technologies.
Zach: Yeah. And it may not even be a monetary divide with the, you know, augmentation class because as the technology gets easier to distribute and more widespread, it's going to become cheaper. So it may be a philosophical divide of people who can afford it but they don't want to live like that. They reject it certain cultures and so the augmentation yeah it's it's it's not necessarily like a wealthy versus non-wealthy thing. It's like which reality like a reality divide.
Emmanuel: Yes. I think you're right. There might be philosophical stance that make people do not accept that. for example, people that do not watch TV, that do not want to use generative AI. But I think that it will take a long time that it will be mostly about money because even if technology is acceptable to be augmented will have a cost and people will not do it for you know for dimes they will ask you for money.
So you at some point you will have to have money to first access the technology and then access to the augmentation itself and then it's it's basically you know most inequalities apart from the philosophical stance are based on monetary differences right either you have access to resources or you not do don't not have access to resources and basically those resources are accessible because you have the money so yes certainly but I don't think that it might be the majority of the cases.
Zach: Yeah. I wanted to ask you about deepfakes because deepfakes are getting crazy and as someone like myself who is not an AI expert, I I'm I'm not able to evaluate the technologies to the level of someone like yourself. like even like I I'm getting things sent to me by, you know, a friend, family member, and like it's to the point now where I'm second-guessing a lot of what I'm seeing because there is crazy real content out there.
You know, there's politicians that say crazy [profanity censored in source] There's Russians that ride bears. like there's there's a lot of real-world things that are completely you know unbelievable in the literal sense as well. So I'm wondering to you as the world of deepfakes get crazier and crazier and the line between the truth and reality start to blur, what does that mean to you as someone who's working in ethics applied to AI?
Emmanuel: It's I feel like it's it's concerning. You've certainly heard about the fact that there has been this kind of deepfake that has been issued quite recently. That was this week about the fact that might they might have a coup military coup in France. Have you have you seen that? No. No. It's just crazy. It has been shared millions of times.
Even there is even an African leader that called the French president to say, "Oh, what's happening in France?" because he saw that on TV and it was a deep a deepfake about a what exactly was the a military coup happening in France that okay the French government has been toppled by the military and it's it's really when you watch it it's it's just crazy realistic that that's just crazy. what I'm concerned about is I would say the schizophrenic stance of countries like France or the US or Germany or the UK saying at the same time okay there are risks about the fakes and at the same time pushing towards more and more artificial intelligence in France like in the US we're pushing towards more what we call AI literacy even the UNESCO the UNESCO the UN is pushing towards you know equality in terms of access to technology and AI which means that if you put lots of people using this kind of system you will have more and more deepfakes right so it's kind of the schizophrenic thing that you are you are you are actually promoting something that you know is really dangerous at the very end of the day so that's the first point the second point is that I fear that we are rediscovering the wheel and maybe you've experienced that you younger than than my than than I am.
But when I was young, I I've always been told when I'm doing studies and when was reading something, do not take anything for granted. Just, you know, cross check with other papers or the news or just make your own conviction based on in inferences that are made out of lots of sources. Right? M the point with AI is that now we feel like AI generative AI will provide us with some kind of solution or truth. So we actually lower our critical thinking. It's it's like I'm it's like if I was talking to you and I was I was like oh because an expert I will have to believe him.
That's not the way it work right when what I'm saying must be questioned. I'm not I do not hold any kind of truth right? I'm I'm certainly making hallucination when I'm talking for sure. I'm I'm certainly saying things that are not correct. So you have to check what I'm saying and you have to make your own conviction. And with AI and especially with the fake because it's more and more realistic, we feel like we can believe it. We don't have to check. We don't have to double check, right?
Zach: The realness of seeing the images and videos of people you recognize on a screen, you know, like that. I I've there are now more and more cases of fraud of people mimicking FaceTime calls or audio calls of you know cloning the voices of relatives and asking them to send money and like I'm in trouble and it's like of course if you have a phone call like that you don't want to have to cross check and be like all right well you know let me call a bunch of pe even though you should like you should you know try to hang up and maybe call someone and double-check and be like, "Hey, is Dave really here?
Is like he not with you anymore? Is he in trouble?" But now you're living in a world where like if you do get a call like that, you might have to go like, "All right, I'm going to hang up on this person in the middle of a crisis and go try to call them from another phone because there's a chance this could be a scam."
Emmanuel: Exactly. So, it makes us really it makes everything very scary.
But what I usually tell my students is that we are rediscovering what we call hyperbolic doubt by Descartes which is that you have to doubt everything even your own senses when you see something you cannot you cannot trust your eyes and that's that's really crazy because you cannot do that all the time or you will turn crazy at some point but each time you interact with an AI system with news I would say that must be the first reflex say, "Okay, is it is it true?" Instead of just taking it for granted and just sharing it on the, you know, social media because lots of people are doing that and spreading something that is a fake news or a deepfake.
Just look at it saying, "Okay, first is it something that is reliable or not?
And what is my interest in sharing something that I'm not sure is true, right? because lots of people are just you know spreading things just for the sake of being present on the social medias right and they're contributing to that sometimes even criticizing it I've seen people that they were sharing deep facing oh look at this deepfake is totally inac no no and you get a big long post on LinkedIn but yes but you're participating to the spread of this kind of [profanity censored in source] if I may say so right so the point is first ask yourself is it reliable is it true and then is it really necessary for me to share it
Zach: Yeah the example you told me of the French coup deep deepfake, which I I'm going to go check it out after this. I'm curious what it looks like, but That is a real-world example of a deepfake creating political mobilization even if it's just a phone call from another leader saying hey is this actually h like it crossed that person's mind to go oh there's a coup going on in France and I'm fast forwarding you know, five, 10 years down the line as deepfakes get better and better. What if it's like something more deadly than that?
What if it's, you know, a someone's able to fake a nuclear launch because they are able to mess with data somehow and there's like this really realistic image of North Korea launching a nuke and then somehow they're able to fudge the data that makes it seem like there's a nuke coming towards France or the US and then that causes, you know, god forbid an all-out nuclear war which would be the end of the world. like that like that seems like a crazy possibility right now, but just like the thing that you're saying with the French coup, like there are realistic near future deepfake scenarios that could trigger real-world crisis.
Emmanuel: That that's for sure. And that's where actually we have to instead of pushing for AI literacy, we have to push for critical thinking literacy. Just think about critical thinking. be more and more critical and also try to I would even say if I if I was honest and I had the discussion with that was the day before yesterday with some of my students about the impact of the news and the social network and just sometimes just get rid of all that because the real-world is not there right just experience the world instead of just watching it through screens because we'll have lots of fakes or fake news this kind thing.
But you're right about the fact that at some point there will be someone that will be using that to create a total panic in France saying okay for example right the Russian have just launched a nuclear weapon something like that and based on the fact that there is a war between the US the sorry Ukraine and Russia that might be something that people would believe. Yeah.
Right. and I and I can even understand what kind of panic might happen even at the highest you know level of the government right maybe they might they might have an overreaction to something that is fake right and then it's something that is totally feasible totally possible totally you know it can happen so once again the point is I'm happy I'm hold because I'm I'm I'm happy I will not see the world turning into something that I do not like but what I'm I'm saying is usually we are living now in a world where we cannot trust anything. It's really sad to me, right? It's really sad to me.
But I think that the first stance would be mistrust and then you trust people or you trust think along the way when you're creating a relation and when you are sure that the legitimacy that the people that are behind the technology or this kind of thing are trustworthy. but I think that the first sense would the first stance would be I do not trust anything. Right? I'm watching that and when I'm when I'm not sure, I will not spread it.
Zach: Yeah, it's I mean, yeah, it it's crazy that there is such this blurred line in 2025 between reality and deepfakes and what people are able to create. It's it's hard to imagine what's what's coming down the pipeline, but yeah, like AI literacy is going to be a huge part of people being able to decipher what is quote unquote real
Emmanuel: And back back to what we were saying, you actually are slowly acculturating people to artificial intelligence. So you make it normal. Yeah. Right. And we see for example that now when you're using for example LLMs or you're using I don't know a generative AI that will that will create new features that know we know that now we have synthetic data which are data that have been created artificially by technology is slowly becoming even bigger than real data.
So even when now you will be prompting an AI system, the system will answer or result will be based on mostly synthetic data things that actually do not exist like the pope wearing this kind of you know remember this image of the pope wearing a really big white jacket or maybe now if you look at the coup in France it's something that in the reality of the LM exists it has been shared millions of time so Now if you prompt an AI system asking have there been a coup in France recently the answer would be yes and it would not be an elucination because it's it's actually coherent with the reality of the data that are used by the LLM.
So you see that with the growing number of synthetic data everything that all the outcomes or the outputs of the of those systems actually they will be less and less trustworthy. Yeah. So mistrust is key.
Zach: True. The going back to the narrative aspect, I wanted to read one more excerpt from that same paper. This is about the new sphere and you write that ultimately to return to thinking the unthinkable, humans may end up in direct connection with the new sphere. N O S P H E R E. a harmonized collective of consciousness similar to superconsciousness, both natural and artificial. This will allow access to an effectively infinite mass of data that individuals will be unable to process efficiently. Humans will be dependent on automated actors who will exploit this data for them and make it readable.
The narratives they convey and promote will be revered as truths regardless of any factual considerations. The human being will no longer be in or out of the loop. They will be no more than an element in a network, a cog in a complex system over which their control will have been reduced to a minimum. Th this goes back to you know sort of like that overwhelming sense of having so much data like you're not able to comprehend it. So you just you lose all ability to find what's true. And that makes me think I if control ultimately belongs to whoever masters the narrative.
If we're in this humans are just a one node in a giant network of the truth that's being fed to them. Will acquiring power in the future be less about force and more about storytelling and narrative control?
Emmanuel: Oh yes. That that's for sure. I would say it's it's maybe already the case, right? Communication, for example, deception has always been kind of a big part of, you know, power and especially in wars, right? We've seen that even in Iraq. so narrative is key. Words are weapons.
I used to say that it goes back to ancient Greece and especially the birth of democracy in the fourth fifth century before current era and especially people like Gorgias saying that actually speech is a tyrant right I mean words are have been always used to influence people and when you master rhetoric when you master speech when you master master words you master the world you know that in communication. We know that in marketing. We know that in politics. We know that in the legal system, right? Words are really weapons of mass destruction.
So if you have a system like AI that is able to spread a narrative widely at a very fast pace, then you must you master the world much more than if you get data because the point is not having data that much is about what you do with it. And if you if you're able to I mean influence the narrative, influence people through the narrative, then yes, you will have a really strong power. And I feel my take now is that this narrative that is conveyed by artificial intelligence that is shaped by what we call norms entrepreneurs is a bigger threat than nuclear weapons.
Zach: You feel the AI narrative being shaped is a larger threat. The nukes.
Emmanuel: Yeah. Wow. Yeah. Because you know with nuclear weapons there is always the fear of deterrence, right? Each time you have to use a nuke, you will say about okay, what will be the retaliation, right? And what is the risk about that? But when you use a narrative, there is no retaliation at all. There is no deterrence against that. So you can use it freely, right? So it's it's and you will touch much more people than than you would use with let's say like on a small nuke sent in France or something like that.
I feel like it's it's it's even more threatening, even more concerning because as you were saying yourself, sometimes it's really subtle. You don't even see it. You don't even think about it, right? So, it's there, it's spread widely, without any kind of, you know, responsibility that is held by any kind of stakeholders. So anyone and you see about the coup, you have to double check that because I'm not sure, but I think that it's a young guy in Africa who did that. I think he's a teenager who did this, you know, fake about the coup in France.
Zach: Does it doesn't surprise me.
Emmanuel: Yeah. Yeah. You see that
Zach: That it sounds like a it sounds like a dare. Like a couple kids in high school like, "Yo, I bet you can't make a video to convince there's a coup happening in the French government."
Emmanuel: Yeah. But you have to double check that. But I think it's it's it's what happened. So I feel like if you master narrative and if you master the medium which is the technology that is carrying the message then you master the world right and you can do whatever you want and we've se we are look look at what we're doing with AI ethics and the way we are shaping the perspective of the whole world about what is acceptable what's not acceptable in terms of ethics applied to artificial intelligence through narrative we've been able to shaped in France I mean in France in the US in Europe at large people are all thinking the same.
It's something that I've seen many times.
I sometime make fun of it because I say that for example when I go to conferences I've been doing conferences for years now on artificial intelligence system when I prompt people like I would prompt AI system meaning when I ask question to people from one person to another I have basically the same answer with the same words with the same narrative with the same ideas and that's really concerning because at some point I mean the norms entrepreneur which is the European Union in terms of AI ethics has done a crazy job changing the mindset of the people, lowering the ability for critical thinking and making people adopt, institutionalize and just repeat like stoastic parrot what they've been told and there you see you see the risk when you have you know male efficient actors that are using this kind of possibility the narrative to shape the mind or influence.
This is the point with deepfakes about the risk on democracy this kind of thing. Then you see the threat, you see what might happen if you're really good at that and you get people that are really good at that.
Zach: So, so what can we do as individuals to fight against the narratives being fed to us or is this just inevitable? Like what options do we have?
Emmanuel: I think it's inevitable but at the same time for those people who have who are willing to do something because lots of people do not care about doing anything, right? I feel like once again doubt is key. Doubt. Doubt. Doubt. Doubt all the time. Doubt about what I'm saying. Doubt about what you see. Doubt about what you read. doubt is really key. Even if sometimes it's really difficult, even sometimes it's it's it's shaking. You have to doubt and it's it's key. And plus what I was saying before also is do not share things that you are not sure of. M critical thinking I think is key.
I mean real critical thinking, not the critical thinking that's promoted by people that are that are not applying it. Yeah. But critical thinking is key. But it's inevitable at the same time because critical thinking requires lots of effort that people are not really willing to make. because most people they do not care. They're just going through life without asking themselves question.
Zach: Fair enough.
Emmanuel: And because psychology is such a complex thing that you get people that are able we call that government mentality which is how to govern people because we know that their psychology can be used to make them they buy something or do something. So you can you can even control behaviors. There are sociological, psychological you know ways to make people do what you want them to do through narrative. For example, what we call per I don't know if you've heard about the performative speech act by John Austin and he wrote he wrote a book which is called the title in English is how to do things with words.
It's performative utterances or performative speech. it's it's when you say something and just by saying it you create a reaction right and I used to say that for example for a like a ripple effect.
Zach: Yes. Exactly. You just say for example the example that he gives usually is that when you baptize a boat right you launch a bottle of champagne on the boat and then you say but actually it's not baptized at all. It's just because you say it that it is baptized, right? So the fact that you are saying things makes things real like artificial intelligence. I used to I use this example with my students. I say AI does not exist. It only exists because John McCarthy 1955 coined the term. But if you So the words speak it into a type of existence.
Emmanuel: Exactly. Exactly. You give birth to something just by naming it. And AI does not exist. That does not exist. You have algorithms. You have programs. That's for sure. But artificial intelligence that does not exist. It's something that has been created by John McCarthy and that's it. Right. But now it shapes the whole way we think about those technologies.
Zach: Right. Yeah, it does. Yeah.
Emmanuel: Let's think about let's think that for example in McCarthy in 1955 for the Dartmouth conference would have called that just algorithms. There are chances that we would not ask the same question as we would not be concerned about the threat by algorithms because algorithms is not considered. The fact that we feel like there is a threat is because we are using the word intelligence and because we have our intelligence and then we have competition between two intelligence and if one is superior to the other then there will be a threat. But if I talk about algorithms no one would care about algorithms. You see what I mean?
Zach: It paints such a different picture in your mind's eye to hear artificial intelligence versus algorithms.
Emmanuel: Exactly.
Zach: And even like if you Google artificial intelligence, it's often this robot with a bunch of numbers streaming past it. Mabel has like some evil eyes. If it's an article about the dark side of artificial intelligence, if you Google algorithm, you're probably just going to get like a bunch of boring code or ones and zeros or like a picture of computer like like it doesn't have the same nowhere near the same like emotive imagery as artificial intelligence.
Emmanuel: That's for sure. That's for sure. But it's exactly the same thing technically speaking. But AI does not exist but it is shaping the way we are interacting with those systems.
Zach: Yeah. Right. I wanted to end off with this because you mentioned doubt and using doubt as a way to fight back against these narratives as an individual. And to me, doubt seems like a very slippery slope because there's there's healthy skepticism, which is questioning things to understand them as a tool. And then if you take doubt to the extreme, it can turn into this whole other thing where distrusting everything becomes your identity.
And I and I feel like a lot of people fell into this trap during co especially in America. inst a lot of people just by nature now they distrust institutions they distrust the pharmaceutical industry even though it's not consistent. People will take Advil but they won't take a vaccine even though it's made by the same company and you know that like it's not a consistent doubt but it's a it's more of like an institutional general fear doubt. So, I'm just wondering like for you, how do you stop yourself from falling into that trap of like staying in the healthy skepticism fear and not going into doubting as an identity?
Emmanuel: I would say that actually doubt once again you have to refer to Descartes there is what we call methodic or methodological doubt, right? So, there is a method for doubting. It's not it's not about being skeptical about everything because you once again you would turn crazy and that does not make really sense.
But when it comes to artificial intelligence and once again the outputs of artificial intelligence the results that you can get from an AI system you have to doubt from a methodic way right you have to double-check you do not have to take it for granted I'm talking basically about artificial intelligence it doesn't mean that you have to doubt about things outside of AI but when it comes to AI it's I think it's it's really key to have this kind of methodic doubt and to Make sure that you will that you will check the sources that you will have different sources v variety of sources to make sure that you get information and not merely data that might be biased data. but yes obviously you know you know Aristotle used to say that virtue is kind of a middle between the two excesses right so you must not fall into one or the other excesses you have to find the right balance and it's up to each individual to make a choice and to take a stance on what they think is the acceptable doubt or even to think that maybe doubt is not totally relevant at all or doubting about everything is relevant.
I'm not a moralist. I cannot tell people what they should say or should they should think or they should do. it's up to each of us to find the right balance.
Zach: Well, Dr. Emmanuel Goffi, thank you so much for taking the time to hop on the show. I really appreciate it. Where can people follow your work? Where can they check you out, follow you on social media? Like what what's the best place for people to keep up with you?
Emmanuel: Yeah, LinkedIn. I'm I'm I'm only on LinkedIn basically. and then obviously if you Google my name they will you will find lots of things but yes LinkedIn is the best place to contact me
Zach: Okay and I'll I'll link that into where people are listening and watching this episode so you guys can go check out Dr. Goffi’s profile and keep up with the work that he's doing there and again thank you so much it was a pleasure this has been an absolute blast
Emmanuel: Thank you thank you so much take care