Latent Space: The AI Engineer Podcast
00:25:01 · Zach Lloyd
What I mean by orchestrator is, like, you need some system that will take the factory definition code and then actually move work along the different stages, calling out to the right models, loading the right skills, etcetera. You need something that will fire the right automations. And then finally, you need a bunch of access points. And again, our philosophy with this, which you can... I could see people have different philosophies. Our philosophy is to sort of bring the factory into a company's existing knowledge work tools. So think of like, you know, the most obvious one is Slack, and like we're doing like 60% of our work is starting in Slack. But sometimes it makes sense to start in the task tracker. Sometimes it makes sense for for a developer who is more comfortable using, like, Claude Code in their terminal to do a bunch of work there and then submit a spec into the factory. And so, you know, our philosophy is to try to to make this whole thing API driven and MCP driven so that the factory itself is like a piece of kinda headless infrastructure where it has all these different entry points. So I think this this stack based view of it when I've shown it or when we talk to companies, seems to land very well. It's like, okay. Yes. This this makes sense. This is a new thing in the, like, developer infrastructure stack. It kinda sits one level back from, like, where CICD is, but I think most companies will have something like this for software development.
00:26:29 · S1, Zach Lloyd
[00:26:29] S1: Yeah. That has a lot of layers, actually. I've never actually seen it spelled out like like you have. I wonder... Because I think most, you know, Cognition and I've... I... You know, Cursor, what... All these words, They all try to do some bundle of, like, the vert... Like, from bottom to top. And there's a little bit of bundling, unbundling with all software. And so there's a critical question of, like, look, I am trying to do my job here as CTO of a, you know, b to b SaaS. Like, I'm... [00:27:00] S2: Do I need like all these, like, six layers? I think it's like, who's this for? Maybe is is the kind of the question. So when we talk to companies that... Where this resonates the most is it has to be a company that has some some amount of scale. Like, this is... If you're like a few people, honestly, this is this is overkill. Yeah. Don't do this. Use the Claude code sub or the codec sub or whatever, and just... You'll go faster. You probably don't need And You could use it. Yeah. [00:27:30] S1: Same for, like, any, like, larger team stack. Right? Like, you... At some point, you switch from, like, fire, YOLO, whatever, to we actually need a platform and system here to accelerate Exactly.
00:27:42 · Zach Lloyd
[00:27:42] S2: Yeah. So you have to be a certain scale. We start seeing that maybe around, like, five to 10 engineers would maybe be the floor where this makes sense. And then all the way up to to very large companies, they want something like this. The bigger the company, the more that this requires, you know, implementation work. Like, I'm sure if you're, you know, you're working with Cognition or or... I don't know if Comment Cursor does, but, you know, to get something stood up like this within a company's own infrastructure, it's not gonna be purely plug and play. There's, like, you know, the sort of complexity of a team's internal developer setup just grows with size. Within, like... But there's a bunch of companies where, you know, maybe, like, it's, like, a 10 to 100 engineers, fast growing, like, growth stage startup, where it is pretty self serve, actually, where it's, like, you just... If they're using, like, the happy stack of, like, Slack, you know, linear Jira... Hang on. Did you call it the happy stack? I called it, like, yeah. I... That's like our... I don't know. No no no shade to to Teams or whatever, like, sure. But, like, there's, like, you know, like, a set of integrations that I find very common at startups where you can plug something like this in and get, like, immediate immediate value without too much setup. I don't know if you... How much you're working like this, but if you're working with, like, Devin or something, you can kinda get some of the same experience where when you're working in a... Some... An environment that's natively multiplayer, so like Slack, for instance, or linear, there's all these benefits that you might not realize just by having, like, this central system of record where you can have, you know, for, like, the Slack integration, for instance, it's like, you can get some flow of information coming in through, like, a server crash channel and some through a user feedback channel. You can just feed that directly into the factory to have it do the work. The factory can loop in multiple people on your team depending on who knows the right part of the code base. And so, you know, I don't think you have to be that big for this for this to make sense, but it definitely doesn't make sense for us...
00:29:48 · Zach Lloyd, S1
For a small team, and it really makes the most sense for a company that's like, I wanna I wanna put in place the right infrastructure so that as models and agents continue to change, we have a system that we're building upon. [00:30:00] S1: There's a lot to go on here. The the the one follow one follow-up question, and then I'm gonna have you plug your AI e talks and just generally where to where to follow-up with you. If you... What is the most steel man criticism of setting up all this? Like, when you encounter criticism or pushback, what should people know as, like, here here is, like, what people are running into? [00:30:24] S2: I'll answer this in, like, there's a certain class of customer where I don't think it makes sense to, like, go on to this particular thing and that's like [00:30:32] S1: Yeah. And beyond the size thing. [00:30:34] S2: Yeah. Where we have challenges right now are companies with very, very bespoke infrastructure or companies with, like, are are kind of not a good fit customer for this, for instance, would be like Uber or Ramp. Like, they're building their own thing. They have very large internal developer teams. But apart from the size thing, I'm thinking like... I mean, there's definitely like the... For some people, they should... It's just gonna be more economical to just use the subs, like, for small... Very small teams, but that comes back to the size Actually...
00:30:34 · Zach Lloyd, S1
[00:30:34] S2: Yeah. Where we have challenges right now are companies with very, very bespoke infrastructure or companies with, like, are are kind of not a good fit customer for this, for instance, would be like Uber or Ramp. Like, they're building their own thing. They have very large internal developer teams. But apart from the size thing, I'm thinking like... I mean, there's definitely like the... For some people, they should... It's just gonna be more economical to just use the subs, like, for small... Very small teams, but that comes back to the size Actually... [00:31:06] S1: Yeah. I don't wanna actually omit the subs stuff because I think a lot of a lot of people go straight to API... Pay by API, and that is between five to 50 times more expensive than just getting a whole bunch of subs. Do people do that? Is is that, like, so against terms of service that, like, just don't bother or, like... [00:31:28] S2: Talk to... Like, a lot of small companies we talk to where they're like, oh, this system seems amazing. We want this, but we're on these, like, collection of $200 subs that we're rotating through. Like, are rotating, [00:31:43] S1: like, 17 different subs. Right?
00:31:43 · S1, Zach Lloyd
[00:31:43] S1: like, 17 different subs. Right? [00:31:45] S2: And then you can't imagine that's allowed, but I I don't... The truth is I just don't really know. But, like, they they are, and it's it's like and it makes... Like, honestly, it's like if you can do that, that's gonna be the cheapest way. But you you you do lose a whole bunch of other benefits if you do that, and it's not a scalable thing. And if your company scales to a certain point, they're... You know, you're definitely not gonna be allowed to do that, but it's like, it it makes sense. [00:32:10] S1: Yeah. I'm just, you know, I'm just saying like, it's... That... That's the extent of subsidy. Like, you get... For like a $200 plan, you get 8,000 of API. Right? [00:32:18] S2: Yeah. It's crazy. [00:32:19] S1: That's not that's not money to sneeze at. [00:32:21] S2: No. No. And it's like, it's one of the... I I mean, just from like a business perspective, it's one of the reasons that, you know, I I don't wanna be in like the pure coding agent business because you you just can't [00:32:34] S1: build Right?