Baseten’s CEO on why every company will want to own its AI

To and for people who don't know what AI inference is beyond it's how you serve the models. I'm really curious to hear you explain it because it feels like the most exciting frenetic part of the AI stack right now. And you're thick in the middle of it with base 10. And I'd love to know, yeah, what it means for people who, yeah, they know the term, but they don't know much else.

Yeah, absolutely. So what is inference? So inference is the way you serve models. Quite often, customers will come to us. They'll say to us, hey, we have this model. We want to use this open weights model. We have post-trained this model. We need to figure out how to actually use it in our applications. And so when you work with a closed frontier model provider, you kind of get this for free. They have the model, but they vertically own everything from the model all the way to the chips it runs on all the way up to the APIs that you are using to call that model. When it comes to post-trained models or your own models or open-weight models, you don't get that for free. And so you'll come to an inference provider and you'll say, hey, I need to run this in production. Here is some scale, approximate scale I need to run this at. And then you'll come to a company like Base10. There is so much that goes into actually running a model. And for us, the infrastructure level problems, so that is like, hey, where is the actual compute going to come from to run this model? So we take care of that. There is the orchestration piece, which is, hey, how do I get this user request to the right chip with the right model on it? We solve that problem. And then there's the chip level details, where it's like, how can I make this model run very, very fast? And so really, when it comes to inference, you're not just looking for an API. You're not just looking for GPUs. You're looking for a performance-reliable system that puts it all together and puts it behind an API. And BaseStand kind of takes care of all that. All right, that was kind of marketing-y, so let's double-click on that. Let's actually double-click on what does that actually mean. So BaseStand's like three different layers of problems that we're solving for our customers. So the first one is this idea of infrastructure and compute. So you need chips to run these models on. So for our customers, if you're a customer running this model at decent scale, you'll probably need thousands of chips to run this. If you go to the market right now and say, I need 1,000 P200s or GP300s or- Good luck. Good luck. It's not happening. Not happening. They're gone. Someone's already taken them. What we do is that we kind of take care of this compute procurement from the standpoint of what you need or that's necessary to run inference on for our customers. We do this using a pretty complex architecture where we're able to pull together compute from all sorts of different regions of cloud. So we sit on top of 20 different clouds in 90 different regions. which is completely abstracted away from our customers. So that's the bottom heterogeneous compute layer. I didn't even know there were 20 different clouds. Yeah, exactly. Well, you call them clouds,