No Priors: Artificial Intelligence | Technology | Startups
00:32:35 · Sarah Guo, Mati Staniszewski
[00:32:35] S0: I see it as. [00:32:37] S1: Mean, the way you think about it, research is head start. This gives us... We can give advantage to the customer earlier, and it's six, twelve months of advantage. That is also a way for us to build a right product layer for you to get best of that research. Frequently, we do that in parallel, so the moment the research is out there, you have the product because we know our initiatives, we know what the product is that's right. So you have research, product in parallel that extends that, but the of the thing that will really give that long term value is the ecosystem that you create around, whether that's the brand and distribution, whether that's the collection of voices you can have, the collection of integrations you can build, the workflows that you can build. I think that's the way we kind of sequence that in our mind, that research product ecosystem that we built, And research, all it is, is head start and being able to accelerate the future a little bit closer. [00:33:28] S0: I think that's a really powerful insight, especially if the research team and the company team believe that as well internally.
00:33:28 · Sarah Guo, Mati Staniszewski
[00:33:28] S0: I think that's a really powerful insight, especially if the research team and the company team believe that as well internally. [00:33:37] S1: I think the piece that was interesting for us is, and I think this is the big questions for all companies that do research and product, is do you wait for research, or do you do a product change? Or even not only research product companies, do you wait for someone else to do the research? Because the timeline for that isn't clear. Is it three months, six months, twelve months? You don't know exactly what it will do, which is the hard choice of do I invest into product layer, or do I just wait more for the research? So in our case, we internally let all the product teams do research initiatives so we can paralyze that work, but we don't hold them that if a product team thinks we should deliver value to the customer by doing something different, they can. And rough rule of thumb is like three months. If we think it's going to be longer than three months, we will probably build it. If it's less than that, we probably won't. [00:34:24] S0: Can you talk about some of the research that you're doing now and then how you think about the cadence of delivery and what's worth working on?
00:34:31 · Mati Staniszewski
[00:34:31] S1: We have now a number of different initiatives across the audio space, and there are kind of two big buckets, and roughly they will relate to that creative and agent side. On the creative side, what this means, we did text to speech models that are controllable. We then added speech to text model that transcribes in a high accurate way, but across a low resource languages as well, so covering almost 100 languages. Then created a music model, a fully licensed music model. And as you think about the future, it's how those models will also interact with some of the visual space. So that's a lot of effort in how you can get the best of audio and then potentially combine that with existing video that you have to really have the best delivery. And then on the agent side, it's of course how you optimize the real time speech to text, real time text to speech. We just released our speech to text model, Scribe v2, which is under 150 milliseconds, 93.5% accuracy across the top 30 languages on FLEURS, and it's only top 30 here because we serve so many others, but most of the people don't, so it's beating all the models on benchmarks. But as you think about the future, it's also the orchestration piece of how you bring speech to text LLM and text to speech. We'll be releasing over the next couple of months a new orchestration mechanism that will lower the end to end part, we think, in a great way, but second thing, which is what is so hard, is not going to only allow you to combine those pieces, but add also the emotional context of the conversation, so you can actually respond with the model, and we think in more expressive and in a better way. And in the future, and something we're investing is parallelizing a speech to speech, more fused approach as well. And of course, depending on the use case, if you are an enterprise reliable use case, the cascaded approach is the approach for the next year to... Has more structure, yeah. More structure, you have more visibility into each of the steps, it's reliable, I'll call tools.
00:36:26 · Mati Staniszewski, Sarah Guo
If you're feeling more expressive and can hallucinate, speech to speech might be the choice, and maybe over time you'll see them kind of go one over another depending on the industry, but that's like a huge investment on our side, which is where the foundation of all the platform and the main part that we are continually investing in is a plethora of different models that combine the best of audio with some of the best of the other modalities together. [00:36:52] S0: I wanna take our last few minutes and ask you a few questions about just the future that I think you'll have a really good point of view on given you think about voice and audio all the time. What do you think of AI companions? [00:37:04] S1: I think they will be a big thing and exist in a big way. Not something I'm personally excited about or something that we spend much time on, but I think the whole line of what's an assistant, companion, character that you enjoy as part of experience were kind of blurry and bland to a large extent. [00:37:26] S0: I think can be very common, but you're not, like, enthusiastic personally about it. [00:37:30] S1: I'm more excited about, like, more of the Jarvis version of that, or more of like I have a super assistant super pilot, kind Versus [00:37:39] S0: the social version.
00:37:39 · Sarah Guo, Mati Staniszewski
[00:37:39] S0: the social version. [00:37:40] S1: Versus the social version. I think it just would be such an incredible unlock, and it's also like it's a... And it's something blending in that personal life context. I would love to start the day, and like someone that understands me, and like start and tell me what's like relevant to me, and open the blinds, and then like tell me about the weather and the sunshine, and play music straight away. [00:38:01] S0: It's gonna happen. [00:38:01] S1: It's gonna happen. That I'm excited for. I think the companion use cases will mention solving loneliness in that part. I think that's one way. Maybe there are like different ways of engaging people back. I do think there will be an interesting future, even if you think about education, where you will have superpower with learning from AI tutors. But I think on the flip side of that, and I think this will, like this is my personal take, you will have education, a good percent of time spent with AI tutors, but then explicit percent of time spent without any technology, human to human, so you can kind of learn that part too. [00:38:38] S0: Yeah, I think this is the correct model, both in terms of emotional guidance and coaching and guardrails, right, as well as like pure pure learning. [00:38:50] S1: Yeah. [00:38:51] S0: What do you think about dictation or what happens in terms of how we control technology that isn't necessarily personified as well? Or does it just all become personified?
00:38:51 · Sarah Guo, Mati Staniszewski
[00:38:51] S0: What do you think about dictation or what happens in terms of how we control technology that isn't necessarily personified as well? Or does it just all become personified? [00:39:03] S1: I think not all personified. I think like some, you know, communicative and often and harm probably will like stay pretty static and [00:39:10] S0: code, I might just [00:39:11] S1: Yeah. Exactly. You don't probably need that much of additional emotional input, I think it's going to be a huge part where, like, in a way, what I hope will happen is you will have ability to stay more immersed in the real life with the devices going back into the pocket, back into some version of an attached element, assuming that's in the right setting, and that kind of acts on your behalf. And in many ways, like, let's say dictation, it's, as Karpathy says, decade of agents, let's call it decade, then you'll have a decade of robots. If you are interacting with robots, of course voice will be the input and the output as one of the key interfaces, so you will need that dictation as huge part. But similarly... [00:39:56] S0: I think the robot's going be personified. [00:39:58] S1: Yeah, relatively. 100. 100%. Yeah, no. I think most of the use cases will be personified. [00:40:04] S0: Okay, last one. What's one thing that you've seen already exist today, or if you project out a few years, will change about how we interact with content? Maybe it's like personalized voice content, or just something people are gonna do with AI voice that they don't do today, or that not everybody knows about.