The Pragmatic Engineer
00:17:58 · Gergely Orosz, Francesco Mancone
[00:17:58] S0: 100 digital products, wow. [00:18:01] S3: Yeah, I mean, it's growing, but it's already like, I think it's an important number. We don't only take care of developing the core features of this product. I think one uniqueness of Bending Spoons is that we built a platform that can provide services that these products can leverage as is, which is also one of our biggest competitive advantage, in my opinion. Just thinking about all the services that we provide from the platform, we're talking about more than 100,000 requests per second. Let's say taking care of everything about authentication, security, monetization, data tracking, all of these utilities that we provide for the units that takes care of digital products is already giving us a lot of traffic that we need to handle that, let's say, pushed us in the years to build complex architecture able to scale accordingly. [00:19:17] S0: Can we talk about your AI products and specifically the AI load there? Because I know Remedy is a very popular AI application, but how does it transform into GPUs? [00:19:29] S3: At the moment, I believe on average daily we do more than 2,000 inferences per second. We can reach peaks of 8,000 per second and that turns into, if you look at the GPU demand that we need to satisfy in order to fulfill these inferences, we're talking about around 4,000 GPUs that we need to allocate Because as you can imagine, they're very pricey and we need to run the infrastructure at the maximum efficiency.
00:19:29 · Francesco Mancone, Luca Ferrari
[00:19:29] S3: At the moment, I believe on average daily we do more than 2,000 inferences per second. We can reach peaks of 8,000 per second and that turns into, if you look at the GPU demand that we need to satisfy in order to fulfill these inferences, we're talking about around 4,000 GPUs that we need to allocate Because as you can imagine, they're very pricey and we need to run the infrastructure at the maximum efficiency. [00:20:10] S1: Not everybody may appreciate how here there's a double problem to be solved. On the one hand, want to deallocate GPUs you're not using so as to save in terms of financial costs. On the other hand, the way the cloud providers operate is that the only way to make sure that a GPU is available when you need it is to keep it booked. There are different ways you can do that depending on the platform, But that also has a cost. And so we've built algorithms that help us predict whether unbooking a particular GPU will result in a comparable GPU not being available or being available. So it's actually a very complicated problem to solve in a way that optimizes for those two contrasting goals of ensuring service and minimizing costs.
00:20:10 · Luca Ferrari, Gergely Orosz
[00:20:10] S1: Not everybody may appreciate how here there's a double problem to be solved. On the one hand, want to deallocate GPUs you're not using so as to save in terms of financial costs. On the other hand, the way the cloud providers operate is that the only way to make sure that a GPU is available when you need it is to keep it booked. There are different ways you can do that depending on the platform, But that also has a cost. And so we've built algorithms that help us predict whether unbooking a particular GPU will result in a comparable GPU not being available or being available. So it's actually a very complicated problem to solve in a way that optimizes for those two contrasting goals of ensuring service and minimizing costs. [00:20:59] S0: And do I understand correctly that, you know, one of the reasons you're doing all of this additional complexity is what you mentioned that you have been profitable and I'm assuming you want to remain profitable because a lot of AI startups will just... They'll just take their funding, VC funded AI startups. They will allocate however many GPUs and their number one thing will be let's get enough users. But sounds like you have another constraint here, right?
00:21:24 · Luca Ferrari
[00:21:24] S1: Yeah, definitely. We have run the company profitably every year initially because 2013, 2014 raising capital in Italy, because that's where we're headquartered, we have people all over the world, particularly Europe, but we started off in Italy. So that's where we had the, you know, the financing questions early on and raising capital equity capital at decent terms in Italy in 2014 was, I would say borderline impossible. And then I think today things are much, much better. I'd like to believe also thanks to Bending Spoons having been a success case and such attracted international capital, but at the time, you know, we didn't have access to it. And also our model is so unusual that we felt no VC would would be interested in investing. So we had to develop that profitability muscle from early on, I think in hindsight, it's been a good thing, we could have probably grown a little bit faster with, with equity financing, but having grown through our own cash flows and a little bit of debt from traditional banks has forced us to be extremely efficient and thoughtful about the allocation of resources, more autonomous and independent. And I believe that scarcity of resources has bred resourcefulness and ingenuity, at least I'd like to believe that's been the case, which today give us an edge over at least some of the competitors, particularly those who have been flooded with financing.
00:23:05 · Gergely Orosz
[00:23:05] S0: Yeah, so I do not frequently hear we took loans from traditional banks with startups. In fact, you're the first one when I hear that. But let's talk about this history because today you said you're more than 400 people, you're headquartered in Milan, Italy, but you started this company eleven years ago. Can you tell me about how it started? And if I recall correctly, when I read your founding story, it started with some AI application ten years ago, not even in Italy. Can you talk about that?
00:23:37 · Luca Ferrari
[00:23:37] S1: Right. When I graduated in Copenhagen, Denmark, studied engineering with two friends at the time. We started a company called Evertail and the vision was to create a self rising diary of a user's life using AI. So we would install this app and it would collect data from the phone transparently, then through AI. And at the time, we weren't even talking machine learning. I think AI in 2010, this is 2010, I believe you only heard about it in academia. I don't think people were talking about it much in the mainstream media. So we built this app, it actually worked reasonably well, but it was a commercial failure. And then on the ashes of that project, we were left with about $40,000 of VC money. In that case, we had raised $1,000,000 or about that in VC money. And when it when we felt we had exhausted all of our avenues and we wanted to shut down the company, the VC firm told us they sold their shares to us for $1 because they didn't want to go through the pretty cumbersome liquidation process. The $40,000 would have been theirs, they had liquidation preferences. But again, for simplicity, they sold us the shares that that amount of money, which was very substantial to us was that was peanuts to them, pretty large fund. And so we ended up taking out this $40,000 from that failure. And that was the seed money for Bending Spoons, which started in Copenhagen in 2013. With that $40,000 the same three people who started Evertail plus two people who had been hired at Evertail and turned out to be particularly capable. And so we asked them to co found Bending Spoons. We started with that amount of money, invested in the first acquisition soon after, I think 2014. So around the same time, we moved the company back to Italy. Again, the time it was just the co founders. And, you know, we invested in the first acquisition $10,000