Getting More Runs from the GPUs You Already Own

Today, compute is like oxygen. It's no longer an option to be multi-cloud. By necessity, you have to do it on day one. If you think about the most important software companies in the last two decades or so, almost all of them are open source. Can you imagine an agent trying to adopt some proprietary? It's just, they just don't do that. We were wrong in building just for compute efficiency early on. Like, wasted GPU hours are expensive, but wasted human hours are even more expensive.

[upbeat music]

Welcome, Zhongheng. How are you today?

I'm good. Thanks for having me.

Thank you for being here.

Oh.

It's been a while.

Yeah, yeah. How are you guys?

Doing well. [laughs]

Excited. Excited for our conversation today.

So I wanted to start and just hear a bit from you, if you can just say what your name is, and in a sentence or so, what SkyPilot does.

So I'm Zhongheng. I'm co-founder and CEO of SkyPilot. Uh, SkyPilot is essentially a platform for all AI compute. Uh, we basically help AI teams bring in their GPU fleets, and we manage them, we help them, uh, uh, get a lot of utilization out of the GPUs. We help their research teams go faster.

You've, you've actually had a much deeper history in research, of course at, at Berkeley, but also in other organizations.

Yeah, for sure. Um, before my PhD, I, I actually worked at Google Brain, but at the time it was very, very, uh, uh, premature, I would say. Uh, super early days. Uh, it's very hard to even get text to speech to work, right? Text to speech meaning, hey, you input, uh, uh, some text, such as a news article, right? And you get speech out, right? Maybe you can listen to it on your, on your commute or something. Um, so Tacotron was really one of the first, uh, effective architectures that, uh, produced really natural-sounding, uh, speech from text. Um, and I was lucky to have contributed a little bit to that research effort.

Tying together your research on Tacotron, and then going over to building and launching SkyPilot, it's interesting that those are both two compute-intensive experiments and research projects to, to have worked on. When you think about that time and how everybody was kind of stuck on work using AWS, and even though they needed this other compute, right, they weren't motivated maybe to, like, figure that out. Or like, what was it specifically, I would say, that, like, caused you to have to solve that problem?

Yeah. Um, I remember at the time when we were in Berkeley, uh, you know, couple years into our PhD, um, we were... The, by we, I mean the co-founders of, of SkyPilot, right? We were actually AI plus system researchers as o- ourselves. We were actually doing AI stuff. And so through, through all these AI projects, my co-founders and I were, like, deep in the weeds of training models, right? Trying to get compute to, to run our jobs. And, and we felt the pain. We felt the pain, uh, uh, firsthand, that you really want to have good software systems that allow AI progress to iterate faster, right? To, to get to that AI progress faster. Um, I think that, that's kind of what motivated us to, hey, let's take a stab at this problem, right? Let's, let's build a system for our own use. And it turns