Roberto Rigobon - Why We're Getting Worse at Measuring

If you can't measure it, you can't manage it. But what happens when the things we measure most carefully are precisely the wrong things? My guest today is Roberto Rigobon, professor at MIT and founder of the Billion Prices Project. He's a super creative economist whose work is influential on the problem of how we actually know what's

happening in the economy. Here's our talk.

Welcome to the Economics of Work. I'm excited to welcome Roberto Rigobon, professor at MIT, founder of the Billion Prices Project, and someone that I think of as, hope this title is okay, as the king of measurement.

Roberto, welcome to the show. That sounds like an SNL, you know, episode. No,

no, it's deserved. So, you know, you can wear the crown. Oh, so thank you, man. Yeah, for sure. No, as someone like with that title, I mean, I think of so much of what we measure in terms of economic statistics as really coming from surveys primarily, at least in the U.S., but most of the government data, most of the economic statistics we rely on are based on surveys. So what do you think is being missed across the board? through surveys being the primary way that we collect data? So I think

the biggest problem we have had are two. One is people have stopped responding to surveys. So, I mean, I am old enough that I remember they used to call my home to get a survey.

Did you participate when they called you?

Well, yes, I participated, but actually my mom was super excited always. Like, oh, they're calling. This must be important. So people responded. Today, the ways of engagement have changed so much and people do not respond. I just wrote a survey. This is among clients of a U.S. association, which is for soybeans. And they gave me about 2,000 close friends. 45 responded. Wow. That's unthinkable. So the first statistical office has that issue. And then the second one is that the economies are transforming the products much faster than before. So when you think about clothing, for example, I mean, the products are churning every four weeks. So whatever method you used to have for surveying, It's not working anymore. I mean, so I cannot imagine people that are estimating the demand nowadays. They will have a nightmare because, I mean, whatever you estimated two months later is useless. So even for the firms to make decisions based on a focus group has to be kind of useless nowadays. So these two things have changed. So we have to change the way we gather the data.

On that first point, if we have declining response rates, in some way you might think, all right, that's fine. We have smaller samples and we just increase the sample size through surveying more or whatever. But it's more problematic if that 45 out of 2,000 that responded are not representative. And as response rates get lower, there's more concern of non-representativeness. It seems like survey respondents really need to... deploy all sorts of sampling weights and like stratification techniques and whatever, you know, like tons of tools, which are kind of the same tools as they would have to deploy if they collected that data, not through surveys. So what is the alternative? Like, how do we get data in alternative ways? And then