00:18:01 · Nikhil Kamath, Mati Staniszewski
[00:18:01] S1: How much does it cost, say, example, how much did Lex and our prime minister... How much did it cost to dub? [00:18:08] S0: You know, this one was harder. In in general, like, if you just wanted to automatically dub, then, like, an hour podcast will be few dollars, up to few dollars. Mhmm. But with with any conversation like like the one of the prime minister, we did employ a wider set of of people that would actually check every translation, with working with the Lex team and the other side of the team to to make sure that the content is preserved, like, we we know... It's like calibre of the conversation where every word matters. So that was maybe a more unique on that side. [00:18:42] S1: We have this use case. We want to dub all the content we make into regional languages, but we've been struggling with solutions really. More than the cost of it, I feel like the emotion behind what is being said is always lost and everything sounds a bit robotic. How do you change that? I've not tried ElevenLabs.
00:18:42 · Nikhil Kamath, Mati Staniszewski
[00:18:42] S1: We have this use case. We want to dub all the content we make into regional languages, but we've been struggling with solutions really. More than the cost of it, I feel like the emotion behind what is being said is always lost and everything sounds a bit robotic. How do you change that? I've not tried ElevenLabs. [00:19:07] S0: Then perfect. It's like almost a cue to show you. No. I think I think that's like one of the big parts that I think we we now can bring to the to the equation that was lost. Like, how do you both preserve the voice, preserve intonation, preserve emotion? And the way, you know, we do it today is we we, of course, detect your speech, we then detect your speakers, we we translate it. But then, with the moment we generate the speech, it actually captures the contextual information from the previous response and tries to recreate that in the in the new target language. Like, effectively, our model, the way we've... We created that for the localization use case, has been trying to match pairs of languages that we have and how would you deliver that in professional setting and how you can reapply that in the automatic DAB so I think it works really well The the the hard things is, like, you know, of course, when you have, like, amplitude of emotions, like Mhmm. Laughter, scream, and you then shift the ordering of the sentence, then it's hard. [00:20:08] S1: Like, see, course. Say, for example, Mati is smiling while you're saying a certain thing. Mati is the right way to say it? [00:20:14] S0: Yeah. Yeah. Yeah. Yeah.
00:20:14 · Mati Staniszewski, Nikhil Kamath
[00:20:14] S0: Yeah. Yeah. Yeah. Yeah. [00:20:15] S1: When you smile when you say a certain thing in English, isn't it hard to take that emotion into Hindi, for example? [00:20:24] S0: The... It's, you know, it's not easy, a 100%. And then there's, like, kind of two... Like, there's the present and the future in the current way, you would still translate it, you would of course dub it. I think you can preserve the emotion, but of course my movement of my lips when I smile will... On the language structure it might be in a different part of the sentence so like maybe an easy example is German from English to German, German usually will have noun at the end so like yes that will exactly happen you will like smile at a different part than the conversation you can do some tricks to try to to still to smoothen it out but yes, there will be some discrepancy so that's present can still work in the future, you know, there are some limitations of the current delivery softwares you'll likely do a lip animation too or lip reanimation to move the lips tiny bit to apply to where you would smile in that part of the sentence so I think that will happen a 100% I hope this happens this year [00:21:30] S1: is that something you would do? [00:21:32] S0: We would want to do it I think like as you think about anything that can amplify the audio experience we want to bring that into our flow. So on that creative side like today if you were to localize hopefully in the next few months you will be able to animate lips straight away as well.
00:21:32 · Mati Staniszewski, Nikhil Kamath
[00:21:32] S0: We would want to do it I think like as you think about anything that can amplify the audio experience we want to bring that into our flow. So on that creative side like today if you were to localize hopefully in the next few months you will be able to animate lips straight away as well. [00:21:48] S1: Could a tiny company be built just around this? Just around changing how the lip moves when something is being dubbed?
00:21:48 · Nikhil Kamath, Mati Staniszewski
[00:21:48] S1: Could a tiny company be built just around this? Just around changing how the lip moves when something is being dubbed? [00:21:58] S0: It could. There's there's actually a good set of companies that are, like, crashing it and going very quickly. Right. You know, of course, think this is an entry point, not the end point. Mhmm. They will likely try to add more of that functionality, but but 100%, you know, it's like... And the the... Of course, now we speak about the static part Mhmm. But the moment you start moving into the real time part, those tiny companies now can be huge companies. And that's kinda like, you know, we spoke about the creative side. If you think about the agent side, and I think there are some use cases here for for for for both the lip animation and even for for for your... For you Mhmm. I have a I have a one idea for you. So, you know, we spoke about the creative side. On the agent side, effectively, what we do is is is allow you to create a voice experience, bring any knowledge that that that you want inside of that experience, and and then deploy that in in in different settings while monitoring, evaluating how it performs. I don't know if you're familiar with Masterclass. Yeah. So Masterclass works with us where they would recreate and work with people like Gordon Ramsay [00:23:07] S1: Mhmm.
00:23:07 · Nikhil Kamath, Mati Staniszewski
[00:23:07] S1: Mhmm. [00:23:08] S0: To create AI Gordon Ramsay that teaches you how to cook while you cook. Mhmm. The top FBI negotiate... Negotiator Chris Voss is part of the ecosystem. You can negotiate and learn negotiating by negotiating with Chris Voss. So you could do your masterclass based on a lot of your existing content, existing work that you already did and and and and spoken about, word about, and actually create a... An Nikhil that you can speak with Yeah. And learn about bootstrapping a $3,000,000,000 company and how to achieve that kind of impact and scale and bring [00:23:41] S1: him Where back would that agent sit? How would somebody find him? [00:23:45] S0: You know, we work with all types of customers, but if you are an individual, you probably would start your own company. Mhmm. Like, we actually have 500 companies that build voice voice agent variations just in India. [00:23:57] S1: Mhmm. [00:23:58] S0: And then now I think over 10,000 across across the world that that that usually they would create their own website or website delivery. But the the the most common now that's kind of shifting is is you you have a phone number that you call and then the voice agent answers. So do [00:24:14] S1: you think there's a market for that if there's an agentic version of myself which has learned from everything I have ever said, all the stupid things and it speaks like me? Do you think somebody will seek it out to interact with it?