Can Open Models Keep Up With the AI Frontier? | Nathan Lambert

When I think about model progress, it comes down to different forms of capital, and that is compute data and talent.

How do you see that playing out in terms of like who's gonna win in the next couple of years?

The Chinese are so good at building models. To say that China is anything but equal on talent, you're gonna have a hard time. What they don't have is compute.

Most people regard distillation as sort of like the main lever for them to do this.

People distill my products every single day. They take it, they strip it down to its raw parts. That's called competition.

Previous post-training was much more reliant on the supervised fine-tuning. Now this has morphed a bit into environments where it's RL heavy. But it's just like hard to map this type of post-training, which is you get the completions and the score goes up to what is going on in reasoning today.

The discourse now is just so much fear on like how much the frontier is just hidden and also breaking away.

The AI capability plot doesn't necessarily worry me, but the AI capability plot in the pretty much non-existing preparation across public cybersecurity and all of these things like-

The frontier AI labs are spending extraordinary amounts of money to build better models. So how does open source keep catching up? In this episode, a16z partner Lisha Li sits down with AI researcher Nathan Lambert, founder of the new nonprofit research lab Trillium, to unpack the race between open and closed AI. They discuss why OpenAI and Anthropic still have major advantages in compute, capital, and speed, but why Nathan believes the technical talent behind open models, particularly in China, is often underestimated. They also separate the reality of model distillation from the rhetoric around it and look at how post-training, data, and RL environments are becoming increasingly important sources of progress. And finally, they turn to recursive self-improvement. Nathan explains why Trillium plans to document how AI is actually being used to improve AI, from the tasks models can already automate to the places they still get stuck, and why making that progress visible could lead to a much more grounded debate about how quickly the frontier is really moving.

So Nathan, really happy to have you here. Thank you for joining in person. Super timely given you're launching Trillium, which I'd love to talk about. But more sort of on, on a macro level, I think we chatted a lot about open source, and that's a lot of what you care about. And what we've been hearing a lot in the public discourse is like, "Hey, we have to pace the frontier, the frontier this, the frontier that." And I think an arc of the stuff that I'm really trying to ask experts about is what is in the frontier? What are we pacing? [laughs] What's there? And then more importantly, kind of threading it with open source and open weight models, like can they keep up with the frontier? And I think this matters very much and is something that you're an expert on. So maybe to sort of start on that arc, w- sort of how you would break down what's gotten us to the frontier for both closed labs and open source models. You know, how does it break down in your mind,