Iran, Surprise UK Elections, AI

Hosted by Christina Ruffini and Sir Richard Dearlove

Guest

Dragos Tudorache

Dragos Tudorache is a European Union MEP and Chair of the Special Committee on Artificial Intelligence and one of the leading architects of the EU's new landmark legislation.

Episode Summary

Former MI6 Chief Sir Richard Dearlove gives his verdict on this week's biggest news stories—the surprise summer general election in the United Kingdom announced by Prime Minister Rishi Sunak and the shock death of Iran's President Ebrahim Raisi and the implications for the regime's succession plan. Julia Macfarlane and Sir Richard also sit down with the European Union's Dragos Tudorache MEP, Chair of the Special Committee on Artificial Intelligence and one of the leading architects of the EU's new landmark legislation.

Listen & Subscribe

• Watch on YouTube
• Listen on Apple Podcasts
• Listen on Spotify

Transcript

INTRODUCTION

Mcfarlane: You're listening to One Decision, the podcast that looks at the choices made that shape our world. I'm your host, Julia McFarlane.

Dearlove: And I'm Richard Dearlove, former chief of British intelligence, otherwise known as MI6.

Mcfarlane: And together, Richard and I speak to the key decision makers, the power players and influencers who shape some of the biggest events that impact us all. Today, we're bringing you a fascinating conversation with one of the key architects of the European Union's recent AI legislation. It is a very good time to be talking about AI disinformation and its impact on all of us, particularly in this bumper year for elections. However, Richard, I first of all have to ask you about one or two really quite dramatic and newsworthy things that happened this week. First of all, we had a very long awaited announcement from the British prime minister, Rishi Sunak.

Sunak: Earlier today, I spoke with His Majesty, the king, to request the dissolution of parliament. The king has granted this request, and we will have a general election on the July 4.

Mcfarlane: So we finally have an election date here in the UK.

Dearlove: Well, I'm very surprised. If you'd asked me to bet on this, I would have lost rather a lot of money, because I think everybody was expecting a November election. And Rishi Sunak actually said earlier in the day, in the second half of the year, but of course it is in the second half of the year by four days, so I thought that was rather amusing. Yeah, everybody thought that Rishi would wait until the last moment. I think clearly what's happened is you have some surprisingly encouraging economic indicators that have come out in the last few days, particularly the rate of inflation. So you have good growth figures, good inflation figures, and they've decided to go for it, but there must be some strategic consideration behind the scenes that I'm not yet aware of in terms of those people who are advising the Conservatives on their election strategy, which gives them, as it were, the momentum to take this decision now. It is almost counterintuitive because everybody thought that the longer they waited, the better the economic argument would be for them. But they've gone for bust and we have an election in six weeks' time, so it's not a very long campaign.

Mcfarlane: I think Labour will have been rather caught by surprise. You're totally right. I think a lot of people have been caught by surprise. There have been some good headlines from the government recently, as you said, on the economy and on inflation. But Sunak is so behind in the polls. Perhaps he might be trying to bet on the summer weather, the feel good factor with the football and the pubs opening late. Possibly he's worried that actually things won't get any better the longer he leaves it.

Dearlove: Well, I guess that could be the case that they saw the situation as getting progressively worse. I mean, the polls have been very discouraging for Conservatives for a long time now and they're supposedly 20 points behind. But I think there are one or two important considerations. The local elections, which of course Labour won handsomely, but they won them on a very small turnout and it's clear that a huge number of Conservative voters stayed at home, so the turnout in most areas was around the 30% mark, even lower. So if you make the calculation that some of the Conservative loyalists will return to their own party with a good set of economic indicators, it may be that, you know, they're going for, well, a small majority, hung parliament, I mean, are all sorts of options. I mean, Labour have got to do incredibly well to win an overall majority because of the redrawing of the electoral map by the Electoral Commission in the UK. And of course, what happens in Scotland is also crucial to the outcome, so it may well be that both parties are thinking that with the SNP in meltdown, which it is at the moment, it's in a real mess, that both parties will pick up maybe more Scottish seats than they expected, although my calculation would be that Labour will win the majority of those seats, particularly the urban ones in places like Glasgow, which were previously held or are currently held by the SNP. Anyway, those are some of the peculiar considerations that we're now facing.

Mcfarlane: Absolutely. Labour's path to victory has to take a turn through Scotland, which is actually where I'm currently talking to you now. It's gonna be a really, really interesting campaign up here, that's for sure. For our international listeners and for our geopolitical watchers, what do you think they might expect if we do get a Labour prime minister on the July 5? What can they hope to see when it comes to foreign policy? I mean, I think we'll likely see quite a lot of strategic continuity. Right? I mean, Rishi Sunak, his first trip abroad as prime minister was to Kyiv to show support for Ukraine, taking some of the line from Boris Johnson. Keir Starmer, he's also been very supportive of the government policy to support Ukraine. The shadow foreign secretary David Lammy, who hopes to become foreign secretary, he's just got back from a trip to Ukraine. And also, there is not a huge amount of daylight between the two main parties with regards to the Israeli Palestinian conflict also, is there? Very little change.

Dearlove: I think you're right, Julia. I think the only difference, the key difference, may be some greater willingness on the part of Labour to relook at the relationship with the European Union. And so the immediate impact might be on the Northern Irish issue, how that relates to Brexit, and also, you know, what the detail of the agreement with Brussels. But I mean, I was listening to an interview this morning where you had, I can't remember which, a senior shadow cabinet minister saying that they would, you know, look at some aspects of the agreement, but they were not going to go back into the single market. So even in that area, I think it'll be subtle changes of detail. The other issue, of course, is migration. They've said that they will repeal the bill that allows the government to send illegal migrants to Rwanda, and they're talking about more European coordination to stop the small boats crossing, but there's so much work already been done with the Europeans on small boat crossings on the channel that I can't see that that's going to be really, you know, the rhetoric may be different, the policy is probably exactly the same.

IRAN AND THE DEATH OF PRESIDENT RAISI

Mcfarlane: And before we move on to our main interview for this episode, Richard, I do have to ask you about some other stunning news we had this week, which came out of Iran a few days ago, which was that the president Raisi and his foreign minister, Amir Abdollahian, were suddenly killed in a helicopter crash after a trip to Azerbaijan. What was your reaction to that shocking news, Richard?

Dearlove: Well, similarly huge surprise. But it's clear that the evidence points to this being a genuine accident, not something more sinister. I mean, they were flying in an American manufactured Bell helicopter, which is 50 years old. If you look at its license number, I think it's actually 54 years old. At the moment, because of sanctions on Iran, it's been impossible for them to get spare parts. So the safety factors relating to this helicopter and helicopters are, you know, well, I wouldn't say they're accident prone, but they're certainly less safe in my view than conventional aircraft. Plus the fact is that they took off in bad weather. I mean it strikes me there's a real element of incompetence about this on the Iranian side as well. Why on earth when you've got a yellow or a red weather warning would you fly an aging helicopter over a range of mountains which are very high? And it looks to me as though they flew into a mountain range and everybody in the plane was killed. But I think I said quite a long time ago, in relation to one of our discussions on Iran, that Iran already had a succession problem. And of course, president Raisi was perhaps one of the leading candidates to replace the supreme leader. So this accident does have a significant political dimension to it and there will be real problems now for the regime. First in the election of Raisi's replacement, but also on the whole question of how the succession is handled. Ironically, it may project the supreme leader's son into his top place, and he wasn't in top place, as a likely successor to his father. The other thing I think which is really important for the Iranian regime is to hold the peace during the sort of process of an extended funeral and, as it were, the build up to a new set of elections. There will be a lot of Iranians who secretly, and I think we've seen evidence of this in social media, are very pleased that Raisi has disappeared. He was, for many reformers, a deeply unpleasant and sinister character.

Mcfarlane: Yeah. I think that's absolutely right. And when he came into power in 2021, he, of course, immediately oversaw that brutal crackdown on those women led protests in Iran. I mean, he has quite a bloody history. He was previously the prosecutor general of Iran, and he was nicknamed the butcher of Tehran for his time on the so called death panel of judges back in the eighties, this panel of judges who sentenced thousands of Iranian prisoners of conscience to death during the purges. But of course, I mean, more recently, he also has the dubious reputation of being the man who launched the first direct attack on Israel in the country's history. And so with him now out of the picture, what direction do you think Iran will go to now? They're obviously going to have elections in a month from now, not exactly the most free or fair elections, but what do you think strategy will be from the supreme leader, from the leadership, from the clerics? I mean, will they want to see someone in the presidency who will help to maybe take the temperature down after Raisi really ratcheted it up after taking over from Rouhani, who was a moderate. I mean, you think we'll likely see a more hardline president who is going to amplify the war rhetoric with Iran's enemies even higher or take that temperature down?

Dearlove: I think that's very hard to predict. My expectation is that the ultra conservatives whom Raisi represented will remain to the forefront, will remain in power, and that they will try to continue down the current track with no significant change. But I think this will come clear over the next six weeks, but my prediction is at the moment no change. I think what is much much more important against this background is the American initiative with the Saudis to try to do a deal which would include eventually Israel and some guarantees for the future of the Palestinian people. I mean I think it just shows in terms of the background to the conflict that Saudi Arabia does not want to see the Middle East dominated by a war caused by the Iranians, and they did, as it were, cause this conflict by pushing Hamas into this invasion of Israel. So I think there are a number of moving parts here, but the most important initiative at the moment is not actually what's going to be happening in Iran, it's whether the Americans can come to some agreement with the Saudis, and whether the Saudis themselves become, as it were, a noble point for, let's say, a restructuring of the Middle East around a relationship between Israel and Saudi Arabia. It may sound extraordinary, but I think that is the picture that we're possibly looking at. It may not be possible to put this agreement together, but it's very interesting that it's in prospect.

INTERVIEW: DRAGOȘ TUDORACHE

Mcfarlane: It's a topic that deserves its own discussion around it. That's for sure. Well, let's move on now to our conversation with Dragoș Tudorache, a Romanian MEP who is also one of the lead authors of the European Union's landmark AI bill. He's the EU's chair of the special committee on AI and really is one of the best voices we could hear from when it comes to global governance on tackling this hugely thorny, complicated, and complex issue that is increasingly going to be a massive part of all of our lives.

Tutorachi: Well, one of the reasons that I jumped in this wagon in the first place was because I was concerned by both, let's call them, the smaller risks, the day to day risks, as well as the bigger ones or what many call the existential threats. And I don't think that focusing either on one or the other or having some sort of a binary approach to it is the right way of looking at it. Because I think AI poses, apart from having huge potential in bringing evident benefits to us all and to many sectors of human activity, social and economical, it does come with some unique challenges, which I think, first of all, have to be recognized. And if you recognize them, then you have a choice. And I think this is ultimately where it boils down in terms of the different schools of thought. It boils down to whether you think that those challenges, those risks are better left to the companies developing or deploying AI to deal with depending on their own set of codes and of conduct or internal constitutions or whatnot. But whether you think as policymaker that you have a responsibility to bring in some clear guardrails to mitigate those risks. And we, as a jurisdiction of the European Union, have made this latter choice where we believe that just voluntary commitments or general principles would not suffice. Whereas other jurisdictions believe that those general principles and voluntary commitments are enough. I'm not judging. I'm just saying that, again, these risks, both the big and the small, have to be mitigated for the benefit of us, individuals, and society at large.

Mcfarlane: Right. I mean, you're essentially talking about the Biden administration's executive orders when you talk about other— Not only. Not only, but it is largely there are basically two big regimes now. There's the Biden executive order and there's the EU AI Act. And you said things like, you know, general principles, voluntary commitments don't go far enough. What the EU has done is it's tried to carry out a sort of a risk based approach to how to regulate AI. I do wanna ask about the differences between the American approach and the European approach. I heard on a podcast that you did recently, when you were talking about why the EU has gone through the risk based approach. And you said that looking at the tech, trying to regulate the tech itself would not be the right way to do this. Therefore, what the EU has decided to do is look at the uses of technology and asked itself, what are those uses that actually bring about risk and bring harms to our values, to democracy and society and things like that? Whereas what the Americans have tried, rather than going on the end use of the product, what the end impact can be, it's tried to start at the other end of the chain and say that we want to determine the principles by which AI is developed. Are you running the risk that things like dual use technologies, things that perhaps are not intended to cause risks, may inadvertently cause risks, given that this is such a rapidly developing thing?

Tutorachi: Well, I don't think so, because it also depends, and one has to look at the way you allocate and deal with the issue of responsibility or liability along the value chains to determine ultimately who bears the responsibility for the actual use and the deployment of a particular application of AI regardless of the intent to which it was developed in the first place. And I think that with the act we also deal with that particular aspect. There was an attempt or a thought initially on the commission side that it will also come with a separate piece of regulation that would deal exclusively with the issue of liability. Somehow we ended up incorporating quite a lot in regulating the chain of liability also in the act, and I think it's the right way to do it. And I wouldn't even say that this is fundamentally the difference between the US approach and the EU approach. I think what is essential to understand with the risk based solution that we have chosen is that we literally looked and we try to isolate what makes in the use of this technology, what makes the most sense to handle from a responsibility point of view. And if you are the one developing it and then putting it on the market, then the responsibility is yours. If you develop an application, but someone else takes it, trains it, directs it in a certain way for a certain purpose, that becomes itself a high risk, then the responsibility moves over to you as the one adding that significant change to the application, and then again deploying it, putting it on the market for a particular use. And in this way, at least we believe in the logic of the AI act, this way we also make very clear who bears ultimately the responsibility for the product or the service going on the market that has an AI built into it.

Dearlove: Dragoș, can I sort of put you a broader issue which is, you know, Brussels, the commission, I would describe as having an obsession with regulation? I mean, is probably one of the most regulatory bodies in every aspect of life down to the noises that lawnmowers make and the shape of trucks and the— And of the cucumber. I'm not going to give the cucumber example. You can understand the EU's approach but I mean in trying to regulate AI or AGI, I have the feeling that Brussels is sort of almost bitten off more than it can chew. And, I mean, what's striking about what you achieved in this first regulatory outcome is its simplicity given that it's a very complex problem but what you've imposed you know from a sort of risk basis is quite straightforward but is this really going to be an effective way? It may give you reassurance in having a piece of paper but is it actually applicable on a day to day basis? Is it a workable solution?

Tutorachi: Well, I trust it is going to be and I would be the first ones to agree with you that if you look at the history of EU regulating, we've had quite a number of bad examples of how you shouldn't be doing it. Not only in terms of how it ended up being implemented, but also in terms of how the EU actually did that regulation in the first place. And you often create more fragmentation, that coherence, and so on and so forth. What I can testify to is the fact that we've been quite haunted from the very start of this work and the negotiations and also before the negotiations when we prepared our mandate. We've been quite haunted by these ghosts of the past, and we really wanted to make sure that we learn as much as we could from those mistakes starting from the way GDPR was put together and many other pieces of regulation, particularly those that were coming close to this idea of regulating the digital tools of today. And, again, we've tried to learn from that and adapt our text and the rules and the regime that we've included in it in a way to make it easier to apply, to make it easier to be understood by those that would be the subjects of the law, and also to make it easier for the authorities that will have to interpret and apply and enforce it to make it so. Is there a guarantee for success in implementation? There is no such thing. You know what they say? The proof of the pudding is in the eating. So we will see what is going to be implemented and enforced, whether this is a bet that we're going to win or not. I'm convinced that we've done the best we could. I don't think literally that you could approach it fundamentally in a different way. It still remains a complex piece of text, but I think that the tools are there to make sure that implementation also happens in the spirit in which we intended it.

Mcfarlane: Well, Dragoș, can I put to you something that causes a lot of people concern is there is an element of the industry and big powerful firms essentially marking their own homework in a lot of this? And I do want to ask you, because Sam Altman did an awful lot of lobbying in the EU last year, and what Sam Altman did was he was lobbying against the categorisation and the designation of OpenAI to one that was a lower ranking of perceived risk. And one can argue that he was successful in that manner. One senior policy advisor at the European Digital Rights and an expert who was consulted on the EU AI Act, Sarah Chander, she says they got what they asked for. She says that the document, the AI Act, shows that OpenAI, like many big tech companies, I'm quoting from her directly now, have used the argument of utility and public benefit of AI to mask their financial interest in watering down the regulation. Isn't a key issue with the EU AI Act the fact that because of how it's constructed, all it takes is for a very powerful conglomerate like Microsoft, which owns AI, are able to recategorise themselves and therefore allow themselves to be out of the purview of some of the most stringent requirements of this act?

Tutorachi: Absolutely not. Why? And I would challenge anyone, including Sarah, to show me which part of the text plays in any way into the hands of Microsoft or OpenAI or which part of the text represents a win for the big tech, literally. Right?

Mcfarlane: Well, they have to face less regulation and less restriction because they are no longer categorized as—

Tutorachi: Who says that?

Mcfarlane: As presenting high risk. There are an awful lot of people who say that ChatGPT-4 and what that represents and the amount of incredible stupendous potential that that has, it does present a risk. And so much of the risk is unquantifiable, and the fact that we don't know how it will evolve.

Tutorachi: We've never said that foundation models such as ChatGPT should be in one of the categories of risk because it's a different kind of technology than the ones that we allocate either as prohibited or as a high risk because they don't have a defined purpose. All the rest of the AI applications that we categorize in the rest of the AI act, we do so by purpose, by objective, by design. Whereas the foundation models, the large language models, they don't have in themselves a purpose that can be put in one category of risk or another. That's why we had to craft a different regime for them. But in crafting the different regime and realizing that they are a different species of AI, we have actually allocated very firm obligations to them. And in fact, ChatGPT and OpenAI are in the higher category of potential risk, which means that they have a higher level of responsibility and longer lists of obligations that they will have to go through compared to other model that would be falling in the lower category because we've ultimately decided to go with a two tier approach. And again, at least judging by the figures of today, OpenAI will certainly be in the top tier category, which means a higher level of responsibility.

Mcfarlane: But it's not at the moment, is it? You would say that OpenAI, and would you say that large language models in general do not belong in the top category of risk?

Tutorachi: No of course they do, that's what I'm saying, they are in that category and when the law will start applying which is one year from now they will be subject of the law.

Dearlove: Do you think that this is a sort of absolutely unique different regulatory challenge? I mean something of a higher order than has ever been attempted before?

Tutorachi: Absolutely. It is, for a million reasons. It is because the technology in itself defies, for many of us defies a lot of everything we've known so far, and it challenges many of the understandings and the prerequisites of everything from creativity to how we do a lot of things. So in itself, it is a groundbreaking technology and it's not easy to pigeonhole it. And I don't think we should. Second, it is a very fast moving technology. So what we had to do in order to still recognizing, and that's why we have those rules in the AI Act, because we recognized it from day one that somehow, yes, of course, they attempted to rid themselves of the need to do anything. But again, they did not win that battle no matter how hard they lobbied. So we did include them in the AI Act because we did think and we do think and we will remain convinced that they have a responsibility in developing those models, which is separate from the responsibility that anyone taking that model and applying it for a given application will have. It's a different set of responsibility that those ones will have. But the developers of these models, of these very powerful models, they have themselves a distinct set of obligations, sorry, of responsibilities. We've recognized those responsibilities. We've attached very clear obligations in the text to them. It's basically around transparency. It's about dealing with risks and having risk mitigation systems in place. It's about reporting incidents and having a mechanism to address those incidents. So it is what we thought is essential to happen for anyone developing and putting on the market a model of this magnitude that has so much versatility in it that can be replicated in a myriad of applications, and where any glitch, any hallucination, any problem in the underlying model would then cascade into problems and hallucinations in many other applications. And also because, again, based on the way these models work, the issue of transparency is also key. Starting from the issue of copyright, which, by the way, we also addressed in the AI Act, we addressed the responsibility for OpenAI and all those to actually be transparent about the copyrighted material used in the training of their algorithms. So again, there is quite a robust set of rules that we've included for the developers of the models. And I fail to understand how anyone would think that they were successful in developing.

Mcfarlane: Can I just go back on OpenAI? I wanna ask about a specific example. So Voice Engine, which is a product developed by OpenAI, it is a text to speech AI model that came out earlier this year. Now, this model uses text input and a fifteen second audio sample to generate natural sounding speech that resembles the original speaker. Deepfakes essentially powered by OpenAI. However, they announced, OpenAI, Sam Altman's firm, announced they would not fully release Voice Engine because it is election year. There are lots of upcoming elections, and there is a potential misuse of the tool for misinformation purposes. So given what you've just told us about how these different companies and the LLMs, how they should be treated slightly differently based on the risk of their use, tell me this, surely this kind of tech, this Voice Engine tech, surely this kind of tech falls into your banned products designation, manipulative AI, regardless of who is having an election soon or not. So on that basis, should Voice Engine never be released?

Tutorachi: Again, I'm not the regulator putting labels, and I don't want to put labels myself on these applications that are about to come on the market or not.

Mcfarlane: But this is an example that tests your regulations?

Tutorachi: No. Again, but I'm not the regulator that is going to put these labels and when they will go on the market. So I can't speak for the regulators themselves. I can tell you what my opinion right now is and how to understand the logic of the categorizations in the tech. So from the little that I understand of this new model that OpenAI has announced, this still is a model. So in the definition of models versus systems and applications in the text, this is still a model. Now, depending on how powerful this model is, it would fall, and I don't know how powerful this model is, it will be either in the top tier or in the lower tier of the models. If it is a model that is on the top tier, then it is going to have the obligations that are specific there. If it is in the lower tier, it's going to have to go through the obligations specific there. In any case, those obligations require transparency for the bottom tier and the top tier. That means that OpenAI will have to show us or show the future regulator how they train the model, what data sets they use to train the model, so on and so forth. How they have designed and instructed the model to work, etcetera. So these are the obligations that are incumbent on everyone that develops and puts on the market any model, top tier or low tier. Now, some of these models, again, I don't know the specifics about this OpenAI tool. Let's say that it is also something that falls under the category of chatbots or authors that are creating content that is artificial, that might also replicate either voice or image or text. And therefore, there are additional obligations for those that will be putting on the market these things, is to label. So you have to label, to watermark the content that is produced by these sort of systems as being generated by artificial intelligence. And this is a way of making sure that me as a customer, when I look at something that comes whether in the form of a voice or an image from any of those tools that are developed, I know that I'm actually looking at myself or I'm looking at an AI generated version of myself. So these are obligations that are specific to those that are falling in this category of chatbots or authors. And then if this particular model that OpenAI has developed is being taken by someone and being used for a specific purpose that is, for example, used to influence or manipulate elections, then you are coming under the high risk category. And then you do have a responsibility also under the high risk list of obligations and the compliance that you have to go through in order to put something on the market or use it for that purpose. Then you also become subject of the law under the high risk application. So it's really, first of all, there's this distinction to be made between the developers of a model versus those that would be using it to develop specific uses for those models.

Dearlove: Yeah, Dragoș, having I discussed this issue with Ro—I'm sure you know Ro Khanna, the democratic congressman for Silicon Valley, who's been heavily involved in the debate in the United States. The line that he was giving to us when we talked to him was that the Americans are pretty fed up that the EU have moved so fast to create this regulatory regime that covers, what is it, four hundred fifty million of the world's population. And therefore this is going to be a very very influential framework. But on the other hand, the Americans feel that given that they're largely responsible, with some notable exceptions, for developing and commercializing the technology, that their thunder has been stolen and they're being forced down a regulatory direction which is being dictated by the EU. So there is a political issue here which is causing, you know, significant disagreement between Europe and the United States and I'm not talking necessarily about which regimes in power, it's a sort of more generalized issue which affects the future of this technology vis a vis how it's used by humanity. And of course whatever happens between the US and Europe will have a massive influence globally, leaving aside what Chinese and the Russians may be doing to pursue their national interests. So I mean given that you've had a lot of dialogue with American side of the equation, and I think with the UK to an extent, where do you think we are in resolving this tension?

Tutorachi: Well, first of all, I wouldn't actually call it a tension. You mentioned it yourself. We've been talking to each other for the last four years at the level of administrations, and there's been the TTC framework that was used for that. Yes. Of course, there were questions. Of course, there were doubts as to whether it is right for the EU to take this approach or not. But all of these questions that popped along the way were resolved through dialogue, through discussions. At the level of the legislators, there hasn't been a single trip that I've done to Washington, where I didn't meet with fellow legislators, in Senate and the House, working or being part of some caucus or working group on artificial intelligence. And the main reason for me making this effort was to make sure that it is well understood where it is that we come from with the legislation. I don't feel that there is a question as to the if, but it's the how. And the fact that there are companies that are upset and companies in California that might be upset and therefore influence the opinion of some policymakers, that's also natural. It's the way it has always been. Now the question is whether we remain forever captive of that opinion. Look at the values that we try to serve, including in the way we look at AI, they're aligned. Look at the main concerns that the US express in official documents coming out of the US, those objectives are aligned. Again, where we still differ, underline the word still. Where we still differ is to how are we going to address those concerns that otherwise we share are out there? In what sequence and how? And I'm also convinced that the political debate in the US will evolve when it comes to AI because society expects things. Society shares the same concerns that we share in Europe. I have seen opinion polls in the US and very serious research done on the kind of issues and concerns that people have when they think about artificial intelligence, they are exactly the same. Why? Because we regard democracy and we have the same values. Again, I am convinced that working together with the US, remaining open to that cooperation, I'm convinced that that debate will also evolve in the US as well.

Mcfarlane: Dragoș, as a policymaker, you must have sat down with countless AI experts who've tried to tell you their concerns, not just the concerns of the populace, you just mentioned. Can I just broaden things out slightly? Because I do want to ask you about Geoffrey Hinton's warning last year, where he talked about rapidly advancing AI technologies. He fears that they could gain the ability to outsmart humans in just a few years' time. And he said that when that happens, AI could evolve beyond humanity's ability to control it. And he gave an example. He said one of the ways in which these systems could escape control is by writing their own computer code to modify themselves. Now, a big issue with this, even with discussing this, is that this is a very difficult, complex issue to grasp, but it is also something that is changing so quickly. And people have referred to this as trying to put restrictions, regulations on a plane that's already in flight. What are some of the fears that need to be discussed more? What things keep you up at night?

Tutorachi: Well, this is certainly one of them. I certainly don't know as much as Sir Geoffrey does about the way these systems work, but from my humble understanding of how they work and how they will potentially evolve, I share his concern of the moment, and I also share his concern that this is not going to be a very long time frame that we have ahead of us until that moment, when we will have to ask ourselves much more profound questions than we had up until today, including, first of all, whether we let the genie out of the bottle or how we put it back in. When it comes to what people refer to as AGI, basically, that ability of an algorithm to actually not necessarily outthink ourselves because some of these algorithms already outthink us. But to basically get that level of consciousness that would make them fundamentally different than what algorithms are today. And that is, or would be a groundbreaking moment, which by that time will have to have us all part of a framework where we can decide together, again, jurisdictions coming together, we have to decide what we do then.

Mcfarlane: But does the regulation mean that we might potentially seed innovation that could help us dig ourselves out of this hole to malign actors like China, for example?

Tutorachi: Well, regulation itself will have to evolve for that moment because regulations, including our AI act right now, is not yet equipped to deal with that particular singular moment when we will be deciding that the genie is out of the bottle. And I also do think speaking of China that in that conversation, we will have to also bring in China around the table because it is a conversation that will be concerning the whole of mankind. So again, we have to prepare. Look at my final speech in the European Parliament just before the vote, I said exactly this out of the many challenges ahead, apart from implementing what we have adopted and many others that have to do with how we bring in society on board of this transformation, how we address the issue of jobs, how we address the issue of education and creativity. So there are many things that are still left to deal with. But the unique challenge that comes from that quantum leap that this technology, I'm convinced, will still make, that moment will have to be accompanied by us being prepared to handle the very difficult philosophical question when that question will come. We will have to all be around the table.

Dearlove: We do have a model. Okay, it's not directly relevant, but you know, the disarmament talks towards the end of the Cold War were, you know, to deal with a technology which, you know, could have destroyed the world. Many people make that association. My own view, and I'd like your opinion on this, is that you made the very clear point that we're not yet equipped on the regulatory side to deal with the final frontier or this fast advancing problem. But maybe this is where there is a technical as opposed to a legal regulatory solution which could be internationally agreed. What I mean by that is that, you know, super advanced AI systems have to have certain algorithms written into them, limit the framework, limit the space within which they operate.

Tutorachi: Yes. And I do think that we need to start already yesterday to prepare for that moment. So I am, in a way, looking forward to the next important meetings that will be taking place, whether in the G7 framework as follow-up to the AI safety summit of last year in London. There are two upcoming ones, one in South Korea that is more of a staging moment for the next one, which will be held in Paris, most likely in early 2025. Basically, we need to be using these upcoming moments to actually start asking ourselves that question. And also prepare for that moment, if I understood your angle correctly, where we say, listen, this is technically a point beyond which we tell ourselves that we cannot allow the technology to pass. Not because it cannot, because the potential will still be there. But we decide collectively as mankind that we don't allow that potential to fully unravel because of the consequences of that unraveling. Consequences which either we find a way to confine to a controlled environment where it does its potential benefits for the good of all. But again, in a controlled environment where we can control the outcome or because we fear the consequences to the point where we'll say, no, you cannot pass beyond that point.

Mcfarlane: But a lot of experts think we are already at that point. Elon Musk is one of them. He signed that letter that said we need to have a moratorium of six months, and he was speaking about ChatGPT-4, about large language models. Yeah. But I think knowing Elon Musk, I think he was just trying to catch up. Because if we look at the current state of the technology as astonishing as it is, it is not there yet. It is not yet a human—

Tutorachi: Certainly, it's not a human conscious. Again, as funny or sometimes inspiring or other times admirable content that might come out of ChatGPT or any other of these models, certainly, you can also get very dumb answers in return. All this to say that we have not yet reached that moment. But again, I think that the potential is there. I don't know whether it's going to be one year or two years or three years or four years. But what I'm saying is that we have a duty of care towards our societies to be prepared for that moment and to signpost that moment in the right way. And I think this is what the next iterations of these fora where our leaders will meet, will need to also address this fundamental question.

Mcfarlane: That was fascinating. I hope you'll come back because there's still so much to discuss, but we're really grateful for your time today.

Tutorachi: No. With pleasure. And it's been a very good talk. I like this chat, so it's a promise. I'll come back.

Mcfarlane: That's it for this week's episode of One Decision. We drop new episodes every Thursday. Like and subscribe so you never miss an episode. Drop us a line. Tell us your thoughts. What decisions have impacted you and where you live? You can write to us. Our email is 1decision@1decisionpodcast.com. From me and the team, thank you for listening, and see you next time.