Business Breakdowns
· Alan Tu
[00:00:00] S1: Totally. I think part of the challenge with Databricks is they actually address so many different use cases. And you may hear folks talk about, "Oh, we use Databricks for recommending movies or pricing strategy or fraud detection," and it's like, "Well, these are not necessarily super related use cases, but it sounds like Databricks is very critical for all these use cases." And then you go a layer deeper and you ask, "Well, what exactly do they do?" And you'll hear folks say, "Well, they process the data." Which then for me lends to the question of, what does "processing the data" mean for someone like myself who does not have a technical background? Even that, it can be hard to grok. And so, the example that resonates for me is that I think we've all had the experience of getting a data file, a spreadsheet in Excel, and you're interested in running an analysis based on that data. And it might be a very simple analysis. It could be something as simple as, "I want to understand the average price of a bunch of different items that were sold." But I think we've all had the experience of, well, you get the data in the spreadsheet, but it's not perfectly set up. It's not every column is exactly where the price should be. And perhaps in certain cells the price is in a certain currency, in another cell maybe it's even the price written out with text. So, you can't just select all that data and just say, "What's the average?" And so, you actually end up spending the majority of your time, sometimes 80% to 90% of your time, just going through the process of unifying all the data into the same format so that you can run that very, very simple calculation of what is the average. So, to me, that pain point of actually getting data into a format that allows you to ask even a simple question is this idea of data processing. And now, in the case of Databricks, just think of that at a completely different scale. You're talking about tons of different types of data sources. There's this concept of unstructured versus structured data. Anything that's rows and columns that fits in a spreadsheet is more structured data. But the reality is most of the data out there is unstructured. It could be log files that are big streams of text, it could be image or video, for folks that are analyzing websites it could be clickstream data.
· Alan Tu
You take that problem of a lot of different types of data formats and how do you get them into a state where you can actually run analysis against it, that's how I would think about the core of what Databricks does. Now they've since expanded and they do all kinds of different things, but that data processing concept is what underpins the primary pain point. And then you can tie that back to all these different use cases. If you think about an e-commerce company that is trying to think about how much inventory should they stock of a particular SKU or a T-shirt, whatever it might be, you could imagine that there's a lot of different inputs that might help you make that decision. It could range from how is your digital advertising performing with that particular T-shirt, it could be how have competitive types of T-shirts been selling, it could be credit card data, it could be all kinds of different data. And if you could get access to that data, you could potentially put that into a process of creating a model to answer that question of, "How many T-shirts should we keep in stock?"
· Matt Reustle
[00:00:00] S0: Makes a lot of sense. I think the painting of the picture of the Excel model certainly will resonate with pretty much anyone. And you can talk to people about how simply taking, whether it's unstructured or just not in the proper format of data, ends up being 95% of the work for what you're doing. And oftentimes, I think that can create this environment where you have the idea of doing more analysis or running a more complicated model, but just that workload up front of doing that stops you and limits ultimately what you're doing. So, I think that actually does bring it to life in a really thoughtful way. I want to go back to the beginning here because a lot of what you mentioned ties back into some of the unique origins of Databricks, but I wanted to just start with the founding team, the academic. And there's all these cliches about academics and them not being commercial, but here you have this really impressive story of evolution. So, can you bring us back to the beginning stage of Databricks, who it was, what it looked like in those beginnings, and tell a bit of the story which I think is really, really interesting for this business?
· Alan Tu, Matt Reustle
[00:00:00] S1: I think it's a very unique part of the Databricks story that to this day, the culture and the DNA of the organization, you can trace a lot of the decisions back to this founding story. And so, it's seven founders that came out of Berkeley. They were all working in what's called the AMPLab at Berkeley around the 2009 time period. And if you roll back the clock to that time period, that was actually the beginning stages of the cloud. And what Ali, the CEO and one of the co-founders, would say is that the seven of them were working in this building together doing research. And actually, on the floor below was another team that was putting out some of the very early research around the data center being the next computer, basically the early concept around cloud computing. And so, you had in one part of the building a lot of innovation around just, cloud computing at the hardware layer, and then you had Ali and his colleagues that were thinking about, "What are some of the software opportunities in cloud?" They were so close to the research around the early ages. And unlike with AI today, where there's been a very clear recognition that AI is going to be a big deal, back then the idea of cloud was still somewhat controversial. [00:00:00] S0: Hard to imagine, but yes.
· Alan Tu
[00:00:00] S1: Totally. This group of folks that were rooted in research gained conviction around the idea of cloud. So then they thought about, "Well, what are the problems that we should work on within cloud?" They actually thought of a few different ideas, but where they ended up was around this idea that data is going to be a really big problem, data at scale. And that if you think about all the different use cases, going back to the beginning of the conversation, there's infinite number of applications around data. The other interesting thing that Ali will say is that during that time period, it was also when Twitter was becoming big and Airbnb was becoming big, Facebook was still becoming big. It was actually a very positive time in technology, and so there was a lot of optimism around entrepreneurship and starting startups. And so, that also fed into the energy of the group. The thought was, well, one of the co-founders was actually the creator of Apache Spark, and they believed that data was going to be an important thing, and so how do we create a business around that? And the other piece that they thought of was using open source as another key bet. And that really was very aligned with this idea of coming from academia and research. When you roll back the clock, there was really three major bets that they had a view on. It was: cloud was going to be big, data was going to be big, and open source was going to be a good way to build a business. In hindsight, it turned out all three of those bets were very good bets. But the idea that at the time was less clear, but because of that environment where there was a lot of optimism around technology, I think all of that coalesced together to be the beginnings of Databricks.
· Matt Reustle, Alan Tu
[00:00:00] S0: On the point of connecting cloud to data, is it fair to assume that by transitioning to cloud, there would actually be more capacity for data to be stored or to be used? Like, is that connected in the sense that prior to cloud, the data capabilities might have been constrained? [00:00:00] S1: Yeah. One of the paradigm shifts of cloud was generally this concept of scale-out architecture, which basically allowed the ability to use more commodity hardware to be able to address larger and larger amounts of both compute and storage and data processing in this case. So, that was an important underlying trend that enabled this proliferation and this idea of just data explosion that I think you're touching on. When you look at what Apache Spark was, it was leveraging this concept of distributed compute and applying that to data processing. That was very important.