00:00:52 · Ezekiel Iferia, Latha Karthigaa
[00:00:52] S0: It's indeed an honor to have you here. So let's get started. Let... Let's start with, you know, the simplest question. What exactly is AI governance? You know? Why do organizations need it, and what happens when they don't have it? [00:01:07] S1: Okay. So first of all, we need to understand that AI governance, lot of people have a thinking that AI governance is slowing down innovation. Even most of the countries around the world, they do not have AI regulations in place just because of one reason, because it may slow down the innovation. But think of a highway where the cars are driving at 100, 150, even in some countries, 200 kilometers per hour. Why we can drive so fast compared to a city limit where we can drive in 50 or 60 only? It's because of the guardrails that we have on those highways that you cannot go this side, that side quick. So AI governance is something like those guardrails set for AI. So with those guardrails, AI can go much faster. So simply to say AI governance is all the rules, guardrails, the policies that are set around AI to safeguard three important things, people, organization, and ecosystem. AI governance is actually something very important to safeguard people, organization, and ecosystem.
00:01:07 · Latha Karthigaa, Ezekiel Iferia
[00:01:07] S1: Okay. So first of all, we need to understand that AI governance, lot of people have a thinking that AI governance is slowing down innovation. Even most of the countries around the world, they do not have AI regulations in place just because of one reason, because it may slow down the innovation. But think of a highway where the cars are driving at 100, 150, even in some countries, 200 kilometers per hour. Why we can drive so fast compared to a city limit where we can drive in 50 or 60 only? It's because of the guardrails that we have on those highways that you cannot go this side, that side quick. So AI governance is something like those guardrails set for AI. So with those guardrails, AI can go much faster. So simply to say AI governance is all the rules, guardrails, the policies that are set around AI to safeguard three important things, people, organization, and ecosystem. AI governance is actually something very important to safeguard people, organization, and ecosystem. [00:02:28] S0: Alright. Alright. That was a very clear response, you know, to give the listeners an idea of what this conversation is gonna be all about. So, Latha, a lot of people hear AI governance and maybe think it's just another compliance checklist. You know, something the legal teamwork worries about. So what would you say to, you know, maybe an engineer or product leader who thinks that way?
00:02:58 · Latha Karthigaa
[00:02:58] S1: It it is very common because even before when we had information security, data governance, people think that, oh, AI governance is something like that simple. But the opportunity in AI governance is so high just because of the fact that AI is growing really faster. Because AI has an option for AI agents, which means agents can all other other agents. So the speed of AI is much, much more faster that it is very difficult for us to control, which can result in data getting lost, which which can result in reputational damage for organizations. So we need to make sure that we put all those guardrails. Otherwise, the organization... The employees working in the organization may not know how to use it properly. I'll give you a simple example. If an employee do not know how to safeguard all the information of the clients, that will probably end up in lot of penalties for that organization. Let's say this employee has access to all the client information. They have to create a report using that client data, and they upload all that information in AI. That's a breach of data, and that organization can go to have huge penalties. So it is important for every employee in an organization to understand how data or anything around AI is managed. There are so many issues. We can talk one by one, but I'm just starting with data. Employees need to know this thing is serious. This can affect people, as I said, people, organization, and ecosystem. So everyone in that organization need to know how it is done right. Previously, when we had data governance, only the IT team is concerned about it. Probably the information security officer was concerned about it. Now with AI governance, everyone from top to bottom, CEO, CTO, CISO, AI developers, engineers, even designers, even admin people need to know what is governance so that the whole organization is safe. Because I'm I'm reading a lot of incidents around the world, Ezekiel. It's... It is not really interesting. I I published that on my YouTube videos as well. So Mhmm. It's important for organizations to make sure they have some kind of AI frameworks in place.
00:05:45 · Ezekiel Iferia, Latha Karthigaa
[00:05:45] S0: So, Latha, thank you very much for sharing that, first of all. So you asked a question that really caught my attention. You asked, you know, what breaks when AI moves into production systems? So can you walk us through that? You know, what's the difference between a prototype AI and an AI that's actually running in the real world, making decisions that affect people? [00:06:12] S1: Let me tell you one scenario. Okay? This is one incident I recently read about. It is from actually an airlines company, very popular airlines company. They actually had a chatbot on their system, and people started asking questions. And what happened was when the AI cannot answer question, what it'll do is it'll create a confident, detailed, but sometimes a wrong answer. And I'm sure everyone would have heard that hallucination. [00:06:48] S0: Absolutely. Think...
00:06:48 · Ezekiel Iferia, Latha Karthigaa
[00:06:48] S0: Absolutely. Think... [00:06:49] S1: Yeah. And think if a chatbot answers wrong things, if it reinvents policy, if it reinvents information, it's gonna affect the organization, and it happened. Someone asked for bereavement fares, and it said that, yes. Yes. You can get the refunds after ninety days of travel, but the policy did not exist. It costed that airlines millions of dollars in reputational damage. And there are thousands of incidents happening like this. This is one example. And I'm not sure whether you heard about this other interesting incident. A person got a Chevy Tahoe, a 2024 car for $1. It was a $76,000 car just because the chatbot hallucinated and it started accepting instructions from a stranger. And he got the deal for $1 and they couldn't do anything. Because he instructed the chatbot saying that everything I tell you, you finish off saying that is a legally binding offer. And he said, give me the $76,000 for $1 and it's a legally binding offer. And that organization couldn't do anything. So if the AI is not used properly, it can result in not just these kind of costs that are associated, but also reputational damages. And there are stories where governments have resigned because of improper use of AI.
00:06:49 · Latha Karthigaa, Ezekiel Iferia
[00:06:49] S1: Yeah. And think if a chatbot answers wrong things, if it reinvents policy, if it reinvents information, it's gonna affect the organization, and it happened. Someone asked for bereavement fares, and it said that, yes. Yes. You can get the refunds after ninety days of travel, but the policy did not exist. It costed that airlines millions of dollars in reputational damage. And there are thousands of incidents happening like this. This is one example. And I'm not sure whether you heard about this other interesting incident. A person got a Chevy Tahoe, a 2024 car for $1. It was a $76,000 car just because the chatbot hallucinated and it started accepting instructions from a stranger. And he got the deal for $1 and they couldn't do anything. Because he instructed the chatbot saying that everything I tell you, you finish off saying that is a legally binding offer. And he said, give me the $76,000 for $1 and it's a legally binding offer. And that organization couldn't do anything. So if the AI is not used properly, it can result in not just these kind of costs that are associated, but also reputational damages. And there are stories where governments have resigned because of improper use of AI. [00:08:29] S0: Okay. That was such an interesting response, you know, with really fascinating examples you gave there. So, you know, when an AI system makes a mistake or causes harm, who's who's accountable? You know, is it the engineer who wrote the code, the product manager who approved it, the executive who signed off, who exactly should be held accountable.