The following is a rough transcript which has not been revised by High Signal or the guest. Please check with us before using any quotations from this transcript. Thank you.

The Next Evolution of AI_ Markets, Uncertainty, and Engineering Intelligence at Scale
===

hugo: [00:00:00] Hi, I'm Hugo Bowne Anderson, and welcome to High Signal. Today, I have the pleasure of speaking with Michael Jordan from UC Berkeley. Mike's work has seriously influenced how we think about artificial intelligence, machine learning, and their real world applications for decades. In this conversation, Mike and I dive deep into the evolution of AI, from its early days to the massive systems driving everything from supply chains to healthcare today.

We discuss how AI is operating on a global scale. Shaping entire industries with its ability to handle vast amounts of data and make real time decisions. Mike shares why the future of AI depends on combining three fields. Machine learning, computer science, and economics. This connection is crucial if we're going to build systems that don't just solve technical problems but also deal with incentives, uncertainty, and human decision making.

One of the biggest themes we explore is uncertainty. How current AI systems [00:01:00] struggle to adapt when things change and why handling this is critical for areas like healthcare where the stakes couldn't be higher. We also talk about the broader impact AI can have on society, including the need for a new kind of engineering discipline.

One that integrates data. Algorithms and human behavior to tackle the challenges we face on a global scale. And of course, we touch on some personal stories like Mike's experience with outdated AI models in healthcare, showing the very real consequences when these systems fail to manage uncertainty.

Before we jump into the full conversation, here's a clip from Mike discussing the evolution of AI and why it's essential to connect the dots between machine learning Computer science and economics. 

mike: A new engineering field is emerging every 50 years, one's emerged, and it's based on the ideas of the last century.

And a lot of the ideas of the last century are statistics, economics, and computer science. Those are fields which emerged and came to some fruition in that era. [00:02:00] One of them deals with algorithms, another deals with incentives, another deals with inference, right, roughly speaking. And there's other related fields, but those are three core ones.

And so you say, well, surely when you build real systems, all of it, you need some inference, you need some algorithms, you need some incentives that you got to bring them together somehow. And so indeed, if you ask about, are there connections between these fields, it turns out there's very good pairwise connections, but not good three way connections.

All right. Let's see. Statistics with computer science, those fields like 50 years ago started to become really emerge and become rigorous and all, and they started to blend and that's called machine learning. That's exactly machine learning. It's algorithms for the service of statistics. So that's, the guy has got conferences and all that and so on.

But the third side, economics, machine learning doesn't have hardly any economics in it. That's been what we've been talking about for the last hour. All right, so, interesting. Now, what about, let's see, statistics meets economics. That's also been existing for at least 50 years. That's called econometrics.

And econometrics is [00:03:00] basically data analysis of economic phenomena. And, but it's passive. You sit there, you analyze the economy. It's like more for macroeconomics. Thanks. Thanks. And you're not trying to build algorithms that do things so much in, in econometrics. You're not trying to do what's called mechanism design in, in, in economics.

So it's a field that's really based on these statistics with kind of causal inference and lots of time series on economic data. So you know, great field. It's in the mechanisms, however. Lastly, uh, computer science with economics. Well, um, that's a thing. That's called algorithmic game theory. Uh, that has its own conferences, its own journals.

It has lots of great people in it. It's been mostly focused on things like auction design. How do you make new algorithms that are better auctions for things to get sold? It has very little statistics in it. Very little inference. Very little data. So the pairwise interactions are very strong. And the last many years of my life, I told a lot of young people, Hey, work on those pairwise interactions.

Those are the fields that are exciting. But now my message is different. If you're not in the middle working on all three [00:04:00] of them simultaneously, you're not really solving the problem. Moreover, I do spend some time in industry. It's very important to me that I go to industry and I watch what's happening.

And when I see real world problems that involve like different kinds of users, different kinds of services, third parties coming into the company and so on and so forth, almost always all three of those elements are literally sitting around the table. There's someone of each of those fields. There's a, and sometimes there's people like in operations research who actually often study all three of those fields that are probably some of the original glue.

But there's people who are self taught, and they can speak the language in all three levels. They're the problem solvers. They're the people who actually do stuff. And so I want our academic world to somewhat reflect that, so I draw this little diagram as a, for the academics to say, don't stay in any one of those corners, all the fun stuff's in the middle.

hugo: And now, before we dive into the interview, I want to take a moment to introduce the amazing team from Delfina, without whom this podcast wouldn't even exist. They've been instrumental in making High Signal a reality. And I'm excited to have them here to talk a bit about their vision and why they're supporting conversations like [00:05:00] these.

So I'm here with Jeremy and Duncan, the co founders of Delfina and producers of High Signal. Jeremy, Duncan, maybe you can tell us a bit about what Delfina does and why we're even doing this podcast. 

duncan: Awesome. Thanks, Hugo. At Delfina, we're building agents for data science, and we've noticed there's a lot of noisy advice in data science.

By nature of our work, we speak with a lot of interesting people in the space, and so with the podcast, we're looking to identify and share the high signal. 

hugo: Well, speaking of interesting people, we just showed a clip from today's episode with Michael Jordan from UC Berkeley. And before jumping into the full interview, there was so much meat in there to grab on.

I'm wondering what resonated with you when Michael Jordan was in particular speaking about the intersection of computer science, statistics, and economics. 

jeremy: So for me, one of the big things was the, the overlap of ML and incentives and the problems that can create. And there's two, two forms of that. One, one is kind of [00:06:00] organizational incentives.

And the second one is more personal incentives and kind of the emotions and ego of the scientists. And the former organizational, we see lots of problems where scientists often will try to apply the most complicated and sophisticated sounding solution to a problem, because that is what. Looks good in a paper or what may, um, get one promoted, but it's often not, um, actually the best, um, solution to the problem at hand.

And we see that over and over again in industry and at Uber. And the second one is around how the emotions of the scientist and the fact that, The building models is gamified by nature. You have an objective function you're trying to maximize. You get very excited when the scores go up. You get depressed when they go down.

And oftentimes that leaves you to chase ghosts and do things that don't actually work. If your metric doesn't actually correctly measure the thing you're trying to improve. And I've seen that myself over and over again. You have to be very careful to check your emotions and assume if things look too good to be true, they probably are.

And Duncan, is there anything you'd like to add there? 

duncan: I love this discussion of the [00:07:00] triangle of professions involved in creating value from data. And I even think, though, that many of the pairwise links are still underdeveloped. And maybe underdeployed. Michael mentions briefly the intersection of computer science with statistics and economics and causal machine learning being one developing piece of that.

And in my experience, causal ML is actually significantly underutilized in industry. In many cases, like most machine learning problems are actually causal in nature. We just don't think of them that way. When we're ranking things on a homepage, the reason we're ranking them is so that we put the things at the top that people will actually click on and buy.

And that's actually a causal problem. You're trying to find the things people will click on, but typically people don't actually model it that way. And finding ways to educate the world about that and take advantage, I think is a huge opportunity for our field. 

hugo: Awesome. I'm so glad those were your takeaways because these are things we [00:08:00] get into a lot more in the interview.

So thank you once again, and everyone time to get into the interview. Hey there, Mike, and welcome to the show. 

mike: My pleasure to be here. 

hugo: I think it's time we talked about artificial intelligence again, or what David Donoho may refer to as recycled intelligence, or some others may refer to as artificial.

Stupidity. I do want to go back to a thought exercise you did at the start of your wonderfully titled essay Dr. AI or how I learned to stop worrying and love economics You started with a thought experiment of if martians came down to earth. What would they think? We seem preoccupied with mimicking human intelligence and you pointed out that we often overlook other types of intelligent systems.

Maybe you could start by telling us a bit about what Martians would conceive of as intelligence. 

mike: I don't know many Martians, but just going back in the history a little bit in the 50s, the phrase artificial intelligence came to the fore. There's this famous conference. And there's this story that people love to tell about the origins, as if that's the origin of everything, and it's just simply not [00:09:00] true.

So around that same time, in fact, the use of that phrase was a reaction to the phrase cybernetics, which was Norbert Wiener, a mathematician at the origin of control theory, signal processing, and a lot of statistics and so on. And so he's a contemporary of John McCarthy and McCarthy didn't like the kind of mathematics that Wiener was doing.

He wanted to do logic. He thought reasoning was logic. And so he wanted to put logic in the computer and that would lead to intelligence. And he wanted a different phrase in cybernetics because they, he just didn't like Wiener. He was going to be a colleague of his at MIT. And so he created this name and it has its appeal, but it does then focus on this ill defined concept of intelligence.

And cybernetics was, you know, a fine phrase. It still gets used from time to time. But in cybernetics, the focus was networks and control systems and not logic, a different kind of mathematics, more statistical. It's actually what's happened, or [00:10:00] at least a closer approximation to what's happened, large networks, collecting things, data, not a statistics, real numbers, no logic in sight, that's really what's happened.

And that's been a success story. So it should have been that the word cybernetics came to describe all of that. But sadly, about five years ago, I think for PR reasons, machine learning was having a success. And I think the people that maybe some of the big companies that were seeing this where this was going to go said, we got to get a better phrase.

The machine learning. So they went back to the older phrase of artificial intelligence and sadly that then brought along a raft of preconceptions. It distorted the dialogue. It also brought along a raft of people, frankly, who weren't doing much for 30 years. Logic based as AI wasn't really going anywhere.

And now those people came to the fore and then they come to the dialogue and I think they confuse it lots of times. The third phrase I want to bring out is something called intelligence augmentation. Uh, and that's a fellow named Douglas Engelbart, he's a little bit later, he's really a computer scientist, he's not a mathematician, [00:11:00] and he also didn't, I think, particularly like artificial intelligence, he didn't like, perhaps, John McCarthy.

They debated, they argued, now they're working on the West Coast, because McCarthy's moved to the West Coast. And Engelbart's point of view was that, no, computers are, we're not trying to necessarily endow them with intelligence or super intelligence and make them, you know, thinkers. If they could do a bit of that, it's great.

But really, the goal is to augment human intelligence. That's what computers really should be aimed to do. It sees the origins of the field of HCI, the mouse, but then like the search engine. Uh, search engine is not itself intelligent per se, but it's using data from all around the world from all kinds of people.

So it's integrating. It's a social phenomenon, a search engine, and therefore it's able to do pretty well. It, it can, it has some forms of knowledge, very implicit, but it's, but the search engine itself, of course, is not intelligent. It doesn't aim to be, and who cares? But it augmented all of our intelligence.

And then we move forward, there was recommendation systems, there was all sorts of things, they become supported by increasingly sophisticated machine learning ideas, and [00:12:00] now we're really in the Douglas Engelbart idea of intelligence augmentation. And I think that's actually what happened mostly search engines plus plus over the last 40 years.

It wasn't the inexorable move towards AI. It was intelligent domination and machine learning came to the fore really more than in the. Norbert Wiener style. And like I say, this was all going kind of great guns, logistic systems, healthcare systems, fraud detection, banks, all kinds of industries were using all these tools.

It was becoming a real thing, all being called machine learning, which is a perfectly fine terminology. 

hugo: And of course, the supply chain stuff with Amazon, you've spoken about before, right? Very serious stuff. 

mike: Very serious stuff. You wouldn't have had a whole raft of industries if you weren't like analyzing data, massive scale with gradient based algorithms that are roughly similar to what we have now.

And doing this all on the proto cloud, not just on one computer, but on tens of thousands of computers, you linked them together. That was the beginnings of the cloud. It was to analyze supply chain data. [00:13:00] And the people doing all that were pretty surprised by how well it was working. It was doing whether the textbook was arguing and it was, it was very important.

During the pandemic, for example, we had. Lots of packages arrived on lots of people's doors. It saved a lot of lives because supply chains were in pretty good shape. And then we also see, when they get a little out of whack, how bad that could be for humanity. Those supply chains are at such a vast scale, there's no way a human or a textbook could run them.

It's all being done with machine learning. And that's the early circa 2000. Then after that, there was continuing development of all these ideas and tools. Great. Including recommendations is where you're bringing in transaction and human data. All right. And it's now leading to yet more commerce and yet more fluidity.

Then at some point it became, let's use the data, not just from transactional behavior or supply chain, let's use data that is language and let's just look at all the language on the web and just bring it into these, you know, now we're using neural nets instead of a random gradient based methods. And let's just see how well that does.

And it does really well and increasingly [00:14:00] surprising how well. At that massive scale, it doesn't break down. It actually gets better and better, but it's all kind of part of a, of an ongoing story. And like I say, this stupid phrase, frankly, artificial intelligence got applied to it a few years ago because it was data from humans.

It was their speech and language. And it looked like it was doing something like a human, whether it is or not, I don't really care. It's doing something interesting and useful in the context of this bigger evolving engineering field. But now the whole dialogue has gotten warped and shifted by these naive science fiction aspirations.

hugo: Absolutely. So in terms of other types of intelligent systems, what are the types? Oh, so I didn't 

mike: answer. Yeah. I didn't answer your question to come all the way back. Yeah. But intelligence, I think in a better phrase for the, I don't have a great phrase, but you know, every 50 years, a new engineering field emerges.

It's roughly, if you go back 50 years, there was chemical engineering, but emerged from the science of chemistry and quantum mechanics. And then 50 years for that electrical engineering emerged from the science of electromagnetism, then civil and [00:15:00] mechanical and so on. So, you know, a kind of a scientific understanding is developed.

It leads to something powerful, and you want to implement it. And it was clear that something was coming probably 50 years ago that the people like, you know, Ravider or, or, uh, John von Neumann, et cetera, saw it. It was information based, it was data based, it was uncertainty management, and it was networks and all that.

It was coming. And critically, it also was economics. So it had microeconomics ideas. Those guys were working on Canero. These were all, they were buddies of each other's. And they, they understood that the local decisions among self interested agents could lead to some global behavior, which was favorable.

That's what, that's what our market is. And I think in their mind, that was part of cybernetics. That was part of, uh, we're, we're, we're building systems where we don't have to put all the intelligence in as a sequence of logical symbols and have a reasoning agent. That's where all the intelligence accrues, but the network itself will be intelligent.

It'll have intelligent behavior. It'll adapt. It'll work over long timespans without anybody worrying about it too much. It'll work at different scales. And concretely, the [00:16:00] classical examples are things like the food that comes into San Francisco every day. No human being is sitting there managing all of that process.

It's rather some person who's not, doesn't even have to use all, most of their brain. They say, I got a lot of tomatoes, and I see over that side of the city there's not many tomatoes. I'm going to drive over there and bring my tomatoes. And if enough people do that kind of thing, by God, supply and demand matches, there's a flow.

And then we have this kind of stable system that every day you go out there, there are tomatoes. And if you didn't build a business on top of that, Relying on the fact that there's going to be tomatoes, the market will provide that for you. And so, again, that works not just in San Francisco, it works around the world, and it works in small cities, it works in big ones, and it's been working for hundreds of years.

If that's not intelligence, if that doesn't have all the attributes of intelligence, I don't know what does. And so intelligence is definitely not a chess player, that is a form of intelligence, not the only kind. So a Martian, I think, indeed, looking at it at a distance would say, I don't understand what's happening on this thing called planet Earth, but I see that we have this problem with food on Mars, and gosh, on Earth, food arrives in all these cities, something [00:17:00] intelligent is happening, and then they'll look about, maybe they'll study it a little bit, and they'll say, oh, I see, these decisions are being made by these entities that we don't really understand very well, and they're making little local microeconomic decisions that leads to this intelligent entity called a supply chain, effectively, or a collection of restaurants.

And, and so someone else could look at that earth and say, that's one kind of intelligence, sure, market intelligence, that may be the first thing they would discover, would be my guess. Someone else could say, well, there's these little entities running around, let's call them humans, and in their brains they got these neurons and they make little local decisions, communicate, it's simply like a market.

And it leads to some other kind of thing, they're able to do other things that aren't just bring food into cities. So, it's a different kind of intelligence. And they would probably look around, there's probably others I'm not even thinking about, right, other forms of ways interactions happen on the planet.

And I think what's exciting is going forward, there will be others and there will be new ones. We have broken democracies right now. We don't interact very well with each other. We've got bad information flows. We're in a prehistoric age, I think, of there could be more healthy, interesting [00:18:00] interactions which should lead to flourishing of art and commerce and democracy.

And we're not even talking about that. We're just talking about, can we get super machines answering all of our questions or destroying us or whatever? The dialogue has just gotten ridiculously warped. And it's, I think it's tragic. 

hugo: Yeah, absolutely. And I really appreciate your heuristic for what intelligence is because it puts aside any questions we have around consciousness and those types of things, which are very, Intangible and difficult to discuss, but having something that's adaptable that works over long and short timescales, that's robust as well, right?

And that's something to your point, we saw during COVID that a lot of our supply chains are incredibly intelligent in their robustness and adaptability as well. And something I've appreciated about your line of thinking is really trying to build machine learning engineering discipline, and not machine learning engineering as we usually use the term machine learning engineering, but we're really talking about principles needed to build planetary scale, as you call it, inference and decision making [00:19:00] I think to figure out where we, to diagnose where we are at the moment, it's interesting to talk about failure modes.

And I know you have a personal anecdote of failure mode from the healthcare system. And I think that might be a good place to start discussing how these systems operate on a global scale. 

mike: Yeah, okay, let me give the example that you're referring to. Also, I want to return to the management of uncertainty, because I think that's a key aspect of intelligence, especially at planetary scale, and how technology could maybe diminish uncertainty, and I think that's really a critical part of this, and again, I don't think we're talking much about that.

We're just talking about these perfect predictions as if they have no uncertainty. But the other, another very important thing, we start talking about long timeframes, like I was alluding to markets working over long timeframes, and that's a part of intelligence. is that I had this anecdote where we, we, um, we, we did an ultrasound.

My wife was pregnant, and we did an ultrasound. And so there's a geneticist in the room, and she said, they were looking at pictures of the baby. And she said, oh, I hate to tell you, but there's little white spots there near the heart. That's calcium buildup, and [00:20:00] that's correlated with Down syndrome. And so that's a prediction, all right?

And, and you can imagine Chagy Vity telling you that. Hey, there's an article out there, and here it is. And so now the question is, how sure are we of that? That really is the question. Because I'm going to, if you're really sure of it, I'm going to do some operation. It was in, in those days, an amniocentesis.

You would go and stick a needle into the womb, and there's a chance you'd kill the baby. And so this starts to get to be meaningful references. And so we're in that situation, and I do what even, what I would probably do this year or two. I said, how sure are you? And she said, there's this literature. And so I went back and dug into it, and long story short, there, there had been studies done.

It was a big, not neural net, but it was a big statistical model with all these features that you measured of the baby and the three dimensional and the blood test and all that. And there was a correlation between white spots around the heart and Down syndrome. And who knows if it was real, but it was there.

There was also something with the nuchal membrane that was correlated, and that turned out to probably be real, and it was used for years afterwards. Right now, [00:21:00] so, at that point, I said, Okay, hmm, uh, there it is. It's a study. But then I thought, I looked, and I realized it was done ten years before. That doesn't mean it's bad, but it was a little old.

And now, critically, the machines that the imaging had been done on for that study, Had a thousand less pixels per square inch than the machine that was used for us. Because the machines had changed, right? 

Of course. 

mike: When you have a thousand pixels more per square inch, there's got to be more white spots.

That's just signal noise in imaging. And, and I said, hey, it could be there's more white spots because the old study is not applicable to the new data gathering process. And so I went back and I said this to the geneticist, she says, oh my goodness, that's interesting. Really, five years ago, we started to have an uptick in the number of Down's syndrome diagnoses.

And around then we got these new machines. That is absolutely 

hugo: wild, isn't it? And it's literal white noise as well. 

mike: It's literal white noise. And I did it back of the envelope calculation. I said, this number of people got that diagnosis that [00:22:00] day. This fraction of them said, I'm going to do an amniocentesis.

And then it was something like, it was some high percentage of those people lost the baby. And if you wrote, did those numbers, ran them on the back of the envelope, you're getting like 100 babies dying a day. And so that happened until finally the technology, some other technologies come along now. You don't need, you can diagnose downs in other ways.

But that, that, so what's the problem there? It's just, it's in some ways obvious. You should have provenance information along with every prediction. Okay, here's my prediction, but it's based on that data. That data is 10 years out of date, and here's the situation. Has something changed that's relevant? All right.

And those are words that a database colleague was looking at. And they'd say, Oh, that's what I might work on. Relevance, provenance, where are things coming from tracking? That's what they did for the banking industry. Well, the banking industry wouldn't have worked on grand scale. If they hadn't worried about all those, they have algorithms that deal with provenance and auditing issues and all that we should be developing all that infrastructure for machine learning predictions, [00:23:00] because if you just run them today, they might be perfectly good.

Cause I got all the world's data for today. It will always be the case that some things will be five years out of date, 10 years out of date, and things will have changed, and you won't know that, and you look at that prediction, you have no doubt what the error bar should be because of the delay in time for the prediction.

And so for building systems that are supposed to work over a stretch of time, we can't always imagine we have the perfect data up to date because it's costly and it just doesn't happen and so on. So that's the kind of infrastructure I want to build as part of this engineering field. And again, I just don't think the dialogue is leading to talking about that infrastructure at all.

It's just saying that, hey, we're going to build a thing on top of the hill, which is going to take in all the world's data, and therefore it'll be perfect. And if it's not perfect, we'll fix it. We'll fix it. We'll fix it. We'll fix it. But that's going to hurt a lot of people in the meantime. 

hugo: It will. And that example I really like for a number of reasons.

One, I didn't actually realize this previously, but it speaks to, as to your point, to drift in a variety of ways. It isn't quite Data drift or quite classical model drift. It's almost telemetry drift, right? Measurement measurement drift or measurement device [00:24:00] drift. But when people think about planetary scale inference and decision making systems, they don't often think of this type of thing, but you've imagined in, in, in your work, I'm going to read you to you.

Actually, we might imagine living our lives in a societal scale medical system that sets up data flows and data analysis flows between doctors and devices. Positioned in and around human bodies, thereby able to aid human intelligence in making diagnoses and providing care. And something underlying this, and I do want to get to uncertainty, but something underlying this is the wisdom of crowds for lack of a better term and how intelligence emerges in collectives.

So perhaps you can speak to that. 

mike: Yeah, the wisdom of crowds is definitely part of this. We are much more wise when we come together than we're individuals. We have this myth of the super, Smart person who could just sit and think and it's smart. That's enough. But if you put any one of us on a desert island and we have to make metal and create fire [00:25:00] and survive and whatever, we're going to all die.

We're just, we're not smart enough to think it through. And it's only when we come together that we start to feed off of each other and ideas start to come out. I actually think that I don't want to bash ChachiBT and LLMs. I actually think they're great. I think that they have a lot of wisdom of crowds in them.

Okay. That's why they're good. If I, in fact, the other day I was trying to fool ChachiBT, I'm giving a talk here in Italy. And I asked if you're in, if you're in Piazza Unita, which is a square in Trieste, I didn't even have to say that. And, uh, can you see the sea? All right. And Chateaupy got the right answer.

I said, now if you're what, if you're looking at the Municipio while you're in that piazza, do you still see the sea? No. It said, no, because now you're turned with your back to the sea, which is really great. And it occurred to me that I happen to know that because I live in Trieste, but. You could ask about any other city in the world, and I won't know.

All right, Chad GPT knows. Why? Is that because it's a super thinker? No, because it used the wisdom of crowds all around the world to have all kinds of little information about now quite fine grained things. Okay, so let's call Chad GPT an artifact, [00:26:00] which is not. artificial intelligence thinking in the computer.

Let's call it mostly wisdom of the crowds. Right? So I want to build on that. I want to go further with that. But there's also wisdom that just isn't a lot of factoids and just stringing together little local assertions and facts. There's other kinds of wisdoms like what's relevant here for this particular decision making in the moment.

And there are things like that. And what's, what am I uncertain about? What am I not uncertain about? That humans are still quite good at relative to where the algorithm machinery is. So I want to have this system that embeds lots of chat GBT kinds of input. Just like search engines are embedded in our world, I want that as well.

But I want to also have it in the context of humans who are, We want humans to do well and put in together an overall system. 

hugo: But had GPT been incorrect, for example, and you'd asked it to express how certain it was of its response, it would have responded in a way. Was it [00:27:00] totally unuseful to you? Right.

mike: It could have again, I, you mentioned, I think the phrase of David Donahoe's recycled intelligence, uh, what it's doing is it's taking old assertions that are in the data of someone asked in some context, how sure are you of that? And some human in the data said, I'm quite sure because blah, blah, blah. It's then generalizing a bit from that, that this situation is similar to that.

Therefore, I'm going to say that you simply have raised it similar to that. And that comes out looking pretty good. And. Lots of situations if you have billions and billions of data points, but I don't think you have to think too hard to think that this situation could be rather different and it's going to require a little, a different context.

And you are going to magnify uncertainty. So I did eventually make tragedy look pretty bad in this regard. I would ask it questions about how sure you about that, about kind of recent events and would say, I'm sure, because here's an article that says it. I said, but there's another article that says something different.

And it says, Oh yeah, that article too. Yeah. Now that you tell me about that, I will change. Yeah. I'm not so [00:28:00] sure. And I could just lead it by the nose and it was clear it has this intelligence, but it also was quite clear that. It was not reasoning under uncertainty at all. It was definitely not. And, and I don't want to interact with an individual who can't firmly tell me that they know something they don't and here's why, and that we can mitigate our uncertainty together.

And ChattiBD, I don't think helps very much with that. And that to me is very problematic right now. That's the current state of the technology. It's going to only get better, but I work a lot on statistical error bars, confidence intervals, and I have algorithms with conformal prediction, others, and my colleagues and I've been working on this for quite a while.

where you can endow an output of a neural network with a little confidence interval. Now, as you alluded to, confidence intervals are very interesting. They could be just a little local uncertainty because, you know, there's some smudge in the data, but they could be also, the data's 10 years out of date, therefore my confidence should get bigger, or because the machine changed, or, really, confidence intervals should be a function of all of the things that could lead to additional uncertainty.

It's not just a simple little statistical formula that you plug in, [00:29:00] right? And that's what we do as humans. We're not perfect at it, but we take all that into account. Oh, that data is out of date. Oh, someone told you that, but they don't, they have a vested interest or whatever. And that leads to adjustment of our uncertainty.

And so that's this interactive big system that I get. I don't want to, you know, an entity, an LLM on top of a hill being fed by all the world's energy and powered. And it supposedly is going to calculate all that. That just is implausible. It's not going to work. It's science fiction. And sadly, it is where much, again, much of the dialogue currently is.

hugo: I love that you've already taken us to error bars and thinking through uncertainty in a more principled way. And I actually want to get to a place where we can talk about your recent ish work on prediction power to inference. But before that, I'm just wondering, Me, a lot of people I know, a lot of people working in data, ML, AI are putting generative AI systems in production currently.

Given the limitations in how they handle uncertainty, what immediate steps do you think people working with data and ML and AI systems can take? to better manage uncertainty in their [00:30:00] models and decisions? 

mike: I don't really have a good answer to that. That's a great question. And I'm sure others want to say better things than I would be able to say.

I just think of myself, it depends on the interaction, but if I'm interacting with a gen AI system, mostly I think of it as an exploratory exercise. I'm trying to explore possibilities and get new hypotheses and try out things. And it'll maybe trigger further thoughts. In statistical inference, which tragedy is part of, it's not outside of it somehow, exploratory is one of the phase things you do.

The next phase, though, is more inferential, what we call inferential in statistics. It's not the inference word that's being bandied around these days. Inferential sort of says, here's a truth in the world, and I want to make sure I cover that truth, that I have a high probability of saying something which is actually what's really happening, a causal inference or whatever.

And then there's another stage beyond even that, which is decisions. And I want to make a decision where in fact, I don't die. I want to make a good decision. And if there's a little error, I want to know about that so that I don't make a decision where the error hurts me. [00:31:00] And if there's an error, I might want to even gather more data.

I want to be in a world where I'm really worrying about that decision. And we're pretty far away from that with these algorithms. So since I know that I'm interacting with chat GPT, I use it as an exploratory device, or if I interact with a generative AI system. It's going to help kick some thinking in and all that.

And I think more or less, most humans are aware of that. They're not trusting this as some kind of a guru that's telling it absolute truth, but it'll maybe take some getting used to feel all you already use this phrase hallucination. And again, I kind of resent when complex phenomena reduced to a single buzzword.

Yeah. And 

mike: hallucination is no, it's sometimes it's just errors. Sometimes it's stuff that's out of date. Sometimes it's just. It's not calibrated well, and there's no answer. Sometimes you, you're not revealing that someone has a vested interest in something, you know, like we do between humans. We know all those things.

And so calling all of that hallucinations, is just damaging to the kind of understanding where we're at and where we should be getting to. And also 

hugo: the other thing that I find personally [00:32:00] infuriating about these systems is they refuse to say, I don't know, they'll make up answer after answer and be like, that's not in my training data.

I can't help you with that there. So I think it's maybe due to the RLHF or maybe DPO or something whereby they're just constantly trying to please you, 

mike: right? They're trying to please you and they will please you as long as you keep asking them and again, pulling them by the nose in some direction.

That's not what you want out of someone who's trying to make a hard decision with you like a doctor. You want somebody who's going to tell you the truth and then also indeed say, I don't know, let's get some more data. Let's ask a second person. Um, also when we interact with each other in natural language, if I ask you something about what's the weather like today in Australia, you'll say various, you'll assertively, you know, well, it's this because I could look outside and see it.

And if I ask you, what's the data, the data like today in Thailand, you'll say, well, I think it's this, but I'm not, you know, the language will sort of convey to me that you're not so sure. Yeah. And you're not embarrassed to say that. And then I will trust you even more in the future. Um, and so in fact, I think I didn't quite [00:33:00] maybe, uh, come to closure on the whole story about the uncertainty.

So we were asking questions about ChatGVT about some recent political events in Italy. And we gave an answer, which asserted too much uncertainty. I'm so sure. And then we said, wait a minute. And then it said, no, I'm not so sure that kind of moved around a little bit. And finally, we arrived at a place that was actually maybe not so bad, uh, and then we did exactly the same thing the next day.

And of course, it had started from zero. Of course, obviously, as a technology person, I know it's not going to adapt all of its weights for that, from that interaction. But I think most people don't expect that out of an agent they're interacting with, which is supposed to be intelligent. They're going to this entity.

It should be listening to them. We're very far, very far away from that, all the state that would be needed. And again, this image of something on a hill, which records all of the state of all the local conversations it's having with billions of people. Again, it's just, it's fine science fiction, and it's kind of dumb.

hugo: Yeah, so I do want to jump now into prediction powered inference. You've done some great work on Prediction powered inference will improve scientific hypothesis testing and in the show notes, I'll link to a [00:34:00] science paper from last year I think in which you and your colleagues and co authors show its applicability to everything from Galaxy classification to estimating deforestation in the amazon not to be confused with amazon.

com of course And counting plankton as well. So i'm wondering if you could tell us a bit about this method how it works and why it's crucial uh I have AI systems that support this type of research. 

mike: Yeah, exactly. Great. We're in the gen AI world now. And so let's take to be concrete. Let's take alpha fold.

So what it does is it generates. Structures of proteins from sequences. That's what, you know, it's one of the big problems in, in, in biological science. And so I would view that as exploratory. It's not telling you, it's not deciding that this protein will work as a drug, if you put it in your body. It's, it's just generating structures.

And now it's doing that based on all the world's data. It's a crowdsourced thing, if you will. And so it, therefore it can be highly accurate. If you hold out some of that data and test it, it'll do [00:35:00] better than any human. It'll do better than any of the previous system by far success. Absolutely. Here's the problem.

If you now use it to do some science, you say, I'd like to do a hypothesis test that, you know, so we did one, is quantum disorder in a protein a good thing or a bad thing, roughly? So, quantum disorder, most proteins like to fold up and become an enzyme and kind of this nice little structure. Sometimes, however, strands hang off of them.

And that's often due to quantum fluctuations, you know, it doesn't fold, so the strands will hang off. So, you could say that's just a broken protein. I suspect that's not going to be used in biology by anything. 20 years ago, some people said, Oh, maybe there's things you could do with those strings. Maybe you could put several of them next to each other and form little grids.

And now you could have kind of a little thing you could put in a membrane and maybe it could control flux in and out of a cell. Turns out that's true, probably. But that wasn't known. And so someone said, let's do a hypothesis test. So they got all the data, the known data, that had been done in labs of sequences of proteins and structures.

So And [00:36:00] whether a protein was phosphorylated or not in a cell. Phosphorylated means it's active. It's really being used for something. So at the end of the day, take all the data they had, which was like, you 000 data points, and you make a little two by two table. Quantum fluctuations, yes or no.

Phosphorylation, yes or no. So that's what a hypothesis test is, like Higgs boson, yes or no, etc. It often comes down to these little kind of assertions. And that little two by two table, you can calculate a little statistic called an odds ratio. If that odds ratio is equal to one, then there's no association between the two variables.

Okay? It means that, it means what it means. If it's bigger than one, oh, there's an association. It looks like that quantum fluctuations and, and phosphorus are associated, which could be interesting. So they wanted to test that hypothesis. Now to test hypothesis, you can calculate the odds ratio based on all that data.

You'll get some number and maybe it's like 1. 3. All right. You now need an error bar to make the decision is you did this exploratory thing. It sounds interesting. It's 1. 3. It's not okay. Maybe I should take this seriously, but a hypothesis test goes further. It [00:37:00] says, what's the error bar. And am I significantly different from one?

Okay. And the answer was the error bar was pretty big and it covered one. So they, an honest statistician say, I can't say, okay, I'm not sure. Let's wait, let's get more data. And that's what they did. 20 years later, uh, another team, uh, says let's redo that, but let's, let's use the alpha fold. Because it can generate not, you know, 10, 000 data points, it can generate 200 million in generative AI, it can generate these things.

So we can now do science with generative AI. We don't have to wait for the lab to generate everything because it's so accurate. It's as if we're doing science with real data, so why don't we just plug it in? So someone said, okay, let's do that. They took 200 million data points. They form that little two book tree table, quantum fluctuations, yes or no, phosphorylation, yes or no, and they calculated an odds ratio, and it was like 3.

5, and they calculated an error bar, because of course they have all this data, quote unquote, and the error bar was tiny. It was like 3. 5, just a little bit above and below, you know. Alright, why is it so small? Because they have 200 million data points. [00:38:00] Now the problem is that, that's wrong. So what happens is that if you actually, so we did a big, We redid the whole experiment, did a Monte Carlo experiment, where you take half of the data, you calculate a ground truth, and then you take the other half and you redo the whole experiment with that, not touching the original.

So the ground truth was like, the number was over here, like two point something, and the error bar was way up here to the right, not covering the truth at all, not even close. Okay, completely wrong, and very sure of itself. All right. All right. That's a problem. You're making a decision based on a wrong error bar.

All right. And so as you alluded to, we did this not just for that domain, we did it for a whole bunch of domains, right? And again and again and again, this phenomenon happened. You take the latest so called foundation model, very accurate overall. But on a particular hypothesis of interest, of real interest, it would be completely sure of itself and completely wrong.

Now why is that happening? We dug into it, it's in some ways obvious. There hadn't been many [00:39:00] studies of quantum fluctuations in proteins. Because that's all, what Alfa Fold was really good at, was all the old data, all the old science. But a scientist is rarely going to go back and query the old stuff. It's, the scientists want to be on the edge of science.

They're asking about something which there's not much data for. And Alfa Fold is going to particularly make errors on that. And it does. And it's very sure of itself. Okay. That's the phenomenon. Okay. So you can say, okay, we will school, write a paper. We can now critique alpha fold. Those guys got it wrong, but that's not our, my goal.

Generally, my goal is to say it's a good tool, but if you use it wrong, it'll make bad mistakes. So let's figure out how to add something that's more statistical on top of it. All right. So long story short prediction power inference says. Uh, do everything you just, I just said, uh, calculate this, uh, use the 200 million data points, but also take a few data points that are gold standard for your particular hypothesis.

You know, maybe a hundred doesn't need much. All right. With those, there's a way we, there's an algorithm [00:40:00] that we can use. It'll take the wrong, highly biased confidence interval and unbiased it. It'll move it over to cover the truth provably that's a theorem. It'll turn out, it'll grow in size a bit too.

That's how it should be. Okay. Thanks. And so when we did that, we got this intermediate technique which had a provably correct error bar, covered the truth, but was much narrower than if you didn't use the alpha fold at all. So it's getting the power of alpha fold, but making a correct inferential decision now, not just a exploratory decision.

And we did that again and again, and we showed that it worked. So we published that in Science, and I think it's getting a lot of attention. Our work often doesn't get a lot of attention, at least in the early days. Later, maybe more. But here, it got a lot of attention right away, because I think a lot of scientists say, Yeah, that's my problem.

And these foundation models, yes, they can be overall very, very accurate, but they can be wrong for the inferential and decision making side. Now, I wish everybody would just appreciate this and all this, but if you pick up, again, the dialogue out there doesn't have any of what I just said in it. It just [00:41:00] assumes that error, small error bars are good, or let's not even worry about error bars because our predictions are so good.

We just got to make the predictions perfect and so on. Self driving cars. Elon Musk, we were going to have self driving cars several years ago because it was just going to make perfect predictions and better than any human. And in some sense that's true, but there's going to be situations it doesn't, it's not exposed to very well.

And it's going to make an assertion, it's going to be dead wrong, and it's going to kill somebody. And, and so what you need there is actually an error bar that it says, okay, here's what I think is happening, but I'm not sure. And so slow down the car, stop it, communicate with some other cars, do, build a bigger system around these things.

Thank you. So anyway, I could go on, but yes, this to me is a, is how technology should go. You think of what is the real problem and not just naive science fiction. 

hugo: Yeah. And I think this is example is it made me think yet again, we have an example in which you mentioned that the reason it didn't perform well is because you weren't looking at old data anymore.

So you have something that's out of distribution, which we [00:42:00] can call data drift or something along those lines, which takes us back to the telemetry drift. Yeah, the health care system that we discussed before. So these failure modes aren't too. Dissimilar. I also love that you mentioned self driving cars or the lack of.

I do think self driving cars, as you've written about before, are a very interesting case of developing, perhaps, artificial intelligence, but the system as a whole mimics some sort of markets more than it does, or supply chains and logistics, more than it does any form of human intelligence, right? Yeah, 

mike: absolutely.

The cars, first of all, should communicate among themselves. They shouldn't be isolated autonomous intelligences. They should be able to say, hey, there's a blockage, there's a kid laying down on the street over there, and the other cars will know that immediately, and, and moreover, there's trade offs. If a lot of cars want to go over there, other cars should know that, so you don't create more congestion, and then there should be a way to manage that, that, hey, I want to go over there, I did, why should you be able to go over there?

That's what we do in real life, there's scarcity, and markets [00:43:00] manage scarcity, and also make things more stable, so I can plan that this is going to happen, because the market will drive that outcome. That's how big systems operate. These autonomous entities that aren't doing any of that are going to make a mess of things.

So it's not just that a single car will occasionally hit somebody while crossing the street. But they will have a very poor market outcome. There will not be a good market. If you don't think about it now, if you think about it, you can make it better. And that's where I do believe things will go. But boy, it's just so many people sitting in those companies aren't doing this style of thinking at all.

They're just doing the ingest lots of data, label it, throw it in there and see if it works. 

hugo: I also want to zoom in on earlier. You mentioned. We have all these conversations around this AI up on a hill and we're not necessarily having conversations around how do we create a context or a playing field, for lack of a better term, in which culture and commerce can flourish.

And with respect to the creation of markets and many sided markets, it does make me think of the work you've [00:44:00] done with United Masters, which is a multi sided marketplace where creators and buyers and that type of stuff, musicians, musicians, Can utilize a platform. So perhaps you could tell us how this plays into all this side, all these ideas.

mike: Yeah. So that's a company that I'm involved with in the U. S. I'm on the board and involved in some of the original scientific development. It's a three sided market. Okay. So we're used to two sided markets often. Where there's a producer and a consumer. And again, a lot of these predictive systems we're talking about, there's not even really a market.

It's just, here's a platform and it provides a service. And it's very implicit whether there's an actual market that you've built. So let's take music as an example, two sided markets I don't think are the right way to think about it. Yes, there's a producer of the music, the person who wrote the songs and sings them, and there's a listener, and you want to connect them up.

So the naive thing to do is just use computer science to connect them up. And, and that's been done very effectively. More songs are being listened to by more people in the world. Things are flowing around and all. In some sense that's great. Me as a listener, I'm [00:45:00] really happy that I can get all this great music, and I get it for free.

Oops, that seems problematic. There's nothing really free in life. Okay, what, what, what's really, what's happening here? Well, we're in a world where, uh, people are not willing to pay for things that are coming in streamed across the internet, because that's how our technology has been developed. Somebody's making some money behind the scenes, but that's not the musician.

And it's someone who created a subscription model or, or advertising, and they're making lots of money. And then maybe they'll send some dribs and drabs back to the musician, but it's not a market economy for the musician. Okay. So, yeah, so could I create something that's better here? Maybe, or maybe the government could come in and tell people that they have to send money to the musician.

That's not how things really work in real life. A three way market can help with this. So United Masters says, okay, I got a bunch of musicians. And, and just to be concrete, there are 3 million young people, mostly, people between 15 and 25, who are signed up as musicians on the, as, you know, they, they're, they didn't sign with a record company, they signed with the United Masters.

And their music is now supplied, and United Masters masters it and sends it [00:46:00] and makes it available, streams it and all that. Um, critically, there's a third leg to this, which are brands. And so brands are something like, so United Masters, the CEO is a friend of mine, he's Steve Stout. He's a legendary hip hop producer in the United States.

He knows lots and lots of famous people. And so he went to the National Basketball Association and said, Folks, I've got these young musicians who are actually Writing some of the songs that many of the young people out there are really listening to. Okay, they put it up on SoundCloud, it gets streamed, and a lot of people are walking around with headphones all day.

They're listening to other young musicians, or young people. They're not just listening to the famous songs again and again. So, when you have neat music on your indie website, behind some clip, Instead of paying Beyonce or Tanya, why don't you use the stream of United Masters musicians? And so now I, as a listener, the third leg of this little triangle, when I go and click a video on an NDA website, I listened to some music and I liked that, maybe I'll do it again.

The musician got paid in that moment, that specific [00:47:00] musician. Now, the other brands, of which there are a couple hundred, Nike or whatever, will look at that transaction and they'll say, I see, that musician is popular on the NBA website. That's a demographic that I now understand. I like those kind of people.

I want to reach out to that musician and say, write some songs for me. Or hey, I'm gonna, I'm gonna stream your music, maybe you have some other songs. The system now has these interactions in all three directions. Uh, where people are incentivized to connect up. And so this is actually working. This is actually a working system.

And now the money is actually flowing because where's the money coming from? It's the brands. And why do the brands have money? Because they got the demographic they're selling other goods to. Moreover, the musician can, I think this is starting to only emerging as right now, because the musician knows who they're connected to in this market.

It's a more transparent market. They can do things like make offers of t shirts. Lots of kids want to wear the t shirt of their favorite musician, not just a Beyonce t shirt. And if there's a site that they can go to and click, and then the t shirt arrives the next day, lots of [00:48:00] brand that would do that, right?

And so, there's an economic system to be built here around local connections among listeners and, and, and, and producers of music. And now, so there are some tens of thousands of young people who are making what's called a salary in this system. And it's, it, the incentives are for everyone to want to participate.

More brands want to get into it. More musicians, more listeners. It's a self building system. If you just have the two way system, you know, person writing music and personal listening that I make some money somehow, there's a temptation to say, well, let's just get rid of the musician altogether. Let's just put gen AI in there.

Indeed. That's what's going to happen. It's going to happen. It's going to kill music to some level. Eventually it'll get, it'll restore. But I don't want to have that happen because there's lots of young people who are ready to have a job being a musician. They're great at it. People want to listen to them.

And so I want to reveal that in a market and not just have some person making a decision that Gen AI will do it and let's see how it works and all that. So I want to have a system where it's all part of that. And I, what I totally firmly believe is that people want to listen [00:49:00] to music written by human beings, mostly.

Uh, cause there's some passion there, there's some life experience. And so I want to make that, all those connections possible. And so the AI is all the data analysis that supports this. For example, that website liked that music. Well, uh, maybe there's other music like that would work there. That's an, that's an AI kind of calculation, the similarity metric in the space of songs.

Also trigger people's interest in writing other songs like that. Trigger interest in people coming to play your party and get paid for that. All these kinds of little local transactions require up to date data analysis and the distribution shifts are all there. And all the data issues are all there.

All the prediction issues. In the context of an overall market. So yes, I could go on as well about that, but I think that replicates, of course, around the world and all the, there's music everywhere, but replicates for other things, other creative acts, even journalism, you can imagine three way markets that are very much, I think this is fun and allows you to be out in the real world, building systems that actually I believe in.

[00:50:00] And it also allows you to do new mathematics and write new kinds of papers that are interesting. That aren't just tweak the old algorithms and then put them out there for the the venture capitalists to pick up on yeah And it 

hugo: does remind me of your work on statistical contract theory, particularly how market mechanisms Can be designed to incentivize certain types of behavior.

So I'm just wondering how you see this framework evolving, particularly, for example, in systems like drug approval or other high stakes industries. So 

mike: I think we're able to get back a little more clearly on about this uncertainty issue. So uncertainty, there's really two ways, two kinds that are, I think are particularly prominent.

One is a statistical uncertainty that I. I have only partial information. I've only gathered a little bit of data and therefore I'm uncertain. If I gathered more, I'd become more and more certain. And eventually I'd be maybe sure there's another kind of uncertainty, which is more economic in origin. And it arises from what's called asymmetry of information in economics.

All right. So, uh, if I want to sell a service to you, I have something to offer. I've got a price that, and, um, uh, I would like, ideally [00:51:00] know what your willingness to pay is for this good that's in economics language. If you were willing to pay a hundred dollars for this good, I would price it at 99. But if I price it at 99 and you're actually not willing to pay that, but you're going to walk away.

So, but yeah, as you can imagine the whole theory, right? So now this is an asymmetry. I'd like to interact with you, but I don't know the right price to set because there's no information you have that I don't have and I will not have. All right. And now if you put that in a bigger economy, there's just all kinds of asymmetries.

And so, you often don't have someone supplying something, doesn't know what's on the other side of the market. And contract theory is a way in economics to mitigate this kind of uncertainty. What you do is you say, I'm going to put out not a single price, I'm going to put out a little menu, what's called a contract.

And that menu will have service price. And there'll be a whole bunch of choices. Right? And, uh, so a classical example is the airplane. There's business class and economy class, and there's a whole range of these things. All of them have their price. That's all adjusted in such a way that there's enough [00:52:00] flow of revenue that the airlines can survive.

Otherwise, they, they were about ready to die because they were putting one price on every seat. That's, when I was young, that's, that, amazingly, that's what the situation was. That's an asymmetry. Uh, the airline does not know when someone walks up to buy a ticket. Are you the kind of person who is willing to pay a thousand dollars for that ticket?

Or are you the kind of person who would not pay more than a hundred? And the issue is, that changes if I, I could be a different person from one day to the next. Because I know in my own head, I really got to get down to Rome today. I'm willing to pay a thousand. Tomorrow I'm not going to be able to.

There's no way you're going to know that. Okay? And these asymmetries will persist, they will change, they will always be there. And so again, the entity on the top of the hill trying to figure out everything and understand everything and make the, make all the right decisions, that, that becomes absurd in that, in that situation.

Okay. So I want to do so statistical contract theory is indeed a blend of economics and machine learning in economics. You would write that contract down typically. Cause you assume you knew various things like the [00:53:00] distribution of willingness to pay, that would be a very common prior to put in. Given you have that, you would then write down some equations and you would get out a contract and it would have provable properties.

In real life, we do not know that distribution of the willingness to pay, but we have experiences. And we start to end that we have biases that we collect data because the willingness to pay can't just be looked at. And that is all part of a machine learning system, but now in the context of a system that deals with asymmetry of information.

Again, no one typically works on this, but this is what I want to push in my academic life is to small work on this, because this is what's needed out there to, to actually have systems that work in the conscious of real human interaction. I want to return just for a moment about this uncertainty. I remember when I was a graduate student, I learned about animal decision making.

So you had a duck who arrives at a pond and you would do an experiment where they want to get breadcrumbs. So there's like a person at one end of the lake throwing breadcrumbs, two crumbs a minute, And a person at the other end of the lake throwing breadcrumbs at one crumb per minute. And [00:54:00] so, you know, so there's a little bit of supply at both ends of the lake, and you throw a duck into that little experiment, and you say, what does the duck do?

A smart duck would go to the place where there's getting more food. They would go to the, you know, the end of the lake where there's two crumbs per minute. And then they would do that every day. That would be the smart thing to do. And so that's called, that's the Bayesian optimal thing to do. Decision making and uncertainty.

What a real duck will do is they will not do that. They will do what's called probability matching. And Two days out of three randomly, they will go over here, but one day out of three, they'll go back over there. All right. So one explanation for that might be, they're not really so sure of things. This is like a bandit problem and machine learning parlance.

They're always exploring a little bit to see if maybe there's something that they didn't think about. But another way to think about this is that what if there's more than one duck? What if there's a hundred ducks that are arriving and all the ducks are being Bayesian optimal and they all go to the place where there's two Brigham's per minute.

Then there's scarcity now, uh, the dogs are not going to feed as well as if they had spread themselves out. How do [00:55:00] they spread themselves out? You can't have a top down administrator tell you, there's got to be some kind of a mechanism. And so probability matching, just randomly, I decide to go there, one out of three, here two out of three, that works.

In fact, mathematically that works, that leads to a social optimum, and it leads to more food for everybody. Okay, so the right decision there actually has to do with the context. Of am I in the world of me all by myself? Okay. Or am I in the world of an economy, effectively, a group of people? And humans evolved to work in groups.

We evolved to deal with these kind of things naturally. And we evolved to mitigate our uncertainty about the situation by being a little bit random and by dealing with it. That, that is a way to think about uncertainty as well. That's a kind, that's a kind of a group based uncertainty versus individual.

Uh, so the error bar there is, uh, very much present in this, but it's not just a simple statistical error bar. So it's a very rich topic, uncertainty quantification, and if you don't do it, you're going to build systems that [00:56:00] really mess up, that really hurt people, that don't deal with scarcity well. Some of the dialogue, unfortunately, goes to the, there's never going to be scarcity again, because AI is going to create so much wealth.

That will not have scarcity. Now we're in the world of science fiction. Sadly though, a lot of the so-called thought leaders in this AI world, they go there very fast. They, they move to the science fiction very fast 'cause they don't really know how to think about this middle build it engineering kind of work.

And 

hugo: there are incentives, of course, for them to do so in, so 

mike: there are, it's funny though that obviously they're making billions and, and maybe that's an incentive. I'm not sure they're necessarily, I know pe civilians, they're not necessarily happy. So happy, uh, they're not so happy with the, with their money.

And I actually don't think a lot of them are incentivized by the money per se. They're incentivized by, they think they're gonna make humanity a better, better by impact and all that. That just the naivete is staggering. And I don't think in any other engineering field, like I alluded to chemical engineering, electrical, and I read a little bit of the history, there was obviously some hysteria, there was some naivete.

But I think [00:57:00] the level of naivete here is staggering. 

hugo: And you've written about civil engineering, the history of civil engineering, all the bridges and buildings that we can build now, right? 

mike: Yeah, I know if you, I would do like to, if I have more time, I'll do more of this, because I think it really is very much what's happening now.

We have an engineering field emerging, and it has properties that come from some of the existing systems. It has similarities to existing economic systems and learning systems, but it's different, and it's going to require its own thing. A name I would probably give to this, if I had to, just pick one, I would call it intelligence engineering.

Chemical engineering could have been called artificial chemistry. That would have been a dumb thing. I think it wasn't called that. Civil engineering could have been artificial civility or electrical artificial electromagnetism or something. Those would have been dumb names and it would have had a bad impact.

Intelligence engineering sounds a little bit dumb right now, but civil engineering probably first rolled it out, but now it sounds great. Um, and I think it might, I [00:58:00] don't like the word intelligence so much because it's so vague. And whereas learning is a little more concrete and error bars even more, but.

If I had to use the phrase intelligence or engineering, it's clear you're engineering. Intelligence, and that could be the intelligence of a system or an economy. It seems a little easier to go from that to, Oh, what are we trying to engineer here? We want some artifact which is intelligent, has intelligent properties.

Uh, and I think, boy, maybe I'm naive, but I just think a little bit of change of that kind of dialogue. Journalists, would they not come in and ask the same dumb questions about, Is AGI gonna happen? Is it gonna destroy? All this dumb stuff. Just a little change of language. Sadly, no one's going to listen to me or not incentivized to do so.

hugo: I do love that you mentioned information asymmetry as well. And I'm going to link to one of my favorite papers that I shared with a lot of people, probably too many people, which is market for lemons, quality, uncertainty, and, and, and the market mechanism, which actually shows under certain conditions of information, asymmetry in markets, you get markets collapsing.

This actually [00:59:00] speaks to your point about two sided marketplaces where platforms can essentially become choke points between. Producers cultural producers and and consumers and not having and have it to your point earlier have incentives to maybe Make just generative AI music. So that one side of the market couldn't collapses as well.

And this speaks to the lack of transparency and data provenance as well. So perhaps we'll wrap up in a minute, but if you can speak to the need for more transparency in markets and for provenance in data flows at a planetary scale. 

mike: Yeah, no, you're speaking my language now. I mean, five years ago, I didn't know about that paper or much economics at all, but I gravitated and I got to a nice read all that.

And I I've learned so much. And if you'll go to my publications page is to put a, I don't like to self publicize, but here I'll do it a little bit. All my papers are about things like market mechanisms and unravel, we have a paper on unraveling, which is exactly that. And in particular, things like data markets.

Suppose I've got a three layer, we call it a three layer market, where [01:00:00] I'm a user, I come in, I get a service from somebody, maybe a banking service. I have to therefore supply data. They give me the service in return, but maybe there's a third entity, which is a data buyer, because the platform doesn't make enough money from the service that they have to get some more money.

So they sell some data to data buyer. Why is the data buyer want data? They want data so they can do market research, whatever, decide whether they want to put a new restaurant in Trieste, which is a Mexican restaurant. They would like to do a little market research and see how many people are talking about Mexican restaurants, what other cities they buy data so they can make those kinds of assertions.

All right. Now let's add privacy to the mix because now we have users. They should care about privacy. Uh, privacy could, let's just say it's differential privacy. So the platforms could say, I will guarantee a certain level of privacy to you. I will add this much noise to all of the data. A different platform's not has enough to add the same amount.

They will add less or more. That's a choice. I will look at those platforms. I'll say, well, that platform over there. [01:01:00] Uh, it's adding more, uh, noise, therefore I'm more private, I'm more protected, I care about that, so I'm going to go to that platform. And maybe a lot of people start to do that, so that platform actually now gets more data.

They could provide a better service, which will make even more people come there. So that's one nice little interaction. But now the data buyers are looking at this and they're saying, well, that platform gives me data and maybe even more than the other platform. But they're adding noise to their data. I don't want noisy data, I want pure data.

So they're a little bit incentivized to not pay as much. All right. And this is a three way interacting layered system. And so there's all these, you can write down equations for all the incentives here, and you can get compatibility. You can get the incentives to interact, and then eventually you get a market equilibrium.

And it has to do not just with classical goods, shoes, and, and socks. It has to do with data and services, but it's all put in this context of this market equilibrium. And now you can ask questions like, well, if you add a, you know, like a regulation on privacy, that there has to be this [01:02:00] level of privacy or there, even if you can't share data at all, put infinite privacy, then how much does that hurt the equilibrium?

Is the resulting equilibrium bad for the users, the platforms, the data buyers? And it turns out you can get answers to those questions, and I'll just hint at it. Uh, if you have two kinds of platforms, what we call low cost and high cost, low cost platforms are ones which need to have this selling of data, because otherwise they don't make enough money, and high cost the, uh, the opposite.

Then, in that market, uh, it is a bad idea to put a, a infinite level of privacy. It hurts the user, actually. Okay? And you will have platforms dying in the middle because of that. And so GDPR, which puts infinite levels of privacy on things, is now hurting small to medium businesses. Not hurting the big businesses, not hurting the small ones, but hurting the middle ones.

And I think we have an explanation for that. But that comes from exactly a study in Markwood Equilibrium in the context of these learning and service type systems. 

hugo: I'll definitely link to your publications page in the show notes as [01:03:00] well. Last time, so, we've been talking about economics and last time we spoke I did tell you I thought the lack of importing tools from economics, microeconomics into data science is kind of in a woeful state and you told me about a triangle of machine learning, economics and computer science, I think.

And I love your insight into how this works and what we can learn from that. 

mike: Yeah, this is just a little suggestion, I usually put this up to help the young people in the audience when I give it a talk, decide what they want to do. Again, I think a new engineering field is emerging every 50 years, one's emerged, and it's based on the ideas of the last century.

And a lot of the ideas of the last century are statistics, economics, and computer science. Those were fields which emerged and came to some fruition in that era. One of them deals with algorithms and other deals with incentives and other deals with inference, right? Roughly speaking, and there's other related fields, but those are three core ones.

And so you say, well, surely when you build real systems, all the, you need some inference, you need some algorithms, you need some, uh, incentives that you've [01:04:00] got to bring them together somehow. And so indeed, if you ask about, are there connections between these fields, it turns out there's very good pairwise connections, but not good three way connections, right?

Let's see statistics with computer science. Those fields like 50 years ago started to become really emerged and become rigorous and all. And they started to blend and that's called machine learning. That's exactly machine learning. It's algorithms for the service of statistics. So that's, the guy has got conferences and all that and so on.

But the third side, economics, machine learning doesn't have hardly any economics in it. That's been what we've been talking about for the last hour. So interesting. Now what about, let's see, statistics meets economics. That's also been existing for at least 50 years. That's called econometrics. And econometrics is basically data analysis.

Uh, economic phenomena, and but it's passive. You sit there, you analyze the economy. It's like more for macroeconomics. And you're not trying to build algorithms that do things so much in, in econometrics. You're not trying to do what's called mechanism design in, in, in economics. So it's a field that's [01:05:00] really based in the statistics with kind of a causal inference in lots of time series.

On economic data. So, you know, gray field. Then we see the mechanisms, however. Lastly, uh, computer science with economic. Well, um, that's a thing. That's called algorithmic game theory. Uh, that has its own conferences, its own journals. It has lots of great people in it. It's been mostly focused on things like auction design.

How do you make new algorithms that are better auctions for things to get sold? It has very little statistics in it. Very little inference, very little data. Okay. So the pairwise interactions are very strong. now. And for the last many years of my life, I told a lot of young people, Hey, work on those pairwise interactions.

Those are the fields that are exciting. But now my message is different. If you're not in the middle working on all three of them simultaneously, you're not really solving the problem. Moreover, I do spend some time in industry. It's very important to me that I go to industry and I watch what's happening.

And when I see real world problems that involve, like, different kinds of users, different kinds of services, third parties coming into the company, and so on and so forth, it's [01:06:00] Almost always, all three of those elements are, you know, literally sitting around the table. There's someone of each of those fields.

There's a, and sometimes there's people like in operations research who actually often study all three of those fields and are probably some of the original glue. But there's people who are self taught and they can speak the language of all three of those. They're the problem solvers. They're the people who actually do stuff.

And so I want our academic world to somewhat reflect that. So I draw this little diagram as a, for the academics to say, Don't stay in any one of those corners that all the fun stuff's in the middle. 

hugo: Totally. And at least anecdotally for me, out of all the tech, big tech companies, I know Amazon actually has the most of that triangle filled out, maybe Uber.

I think so, but I think 

mike: I give credits to say, even places like Uber early on, they built a two way market. That was a real market. There was producers and there was consumers, drivers and writers. And they had a real economist and machine learning people. They built a system, but it was just a two way market.

As I alluded to earlier, that's probably not enough. And moreover, they didn't really know how to price it very well, and they tried to do predatory [01:07:00] pricing using their VC money. And so people were real impressed by it, but it really did have a, I think, an impact on social welfare. A lot of people got rides, uh, they never would have gotten otherwise.

They had to call a taxi while it's snowing in some part of some city, you're out of luck. Uh, Uber has, has definitely helped social welfare. Uh, so I, I would say that a lot of those companies that, and notice that Uber didn't have to do advertising. You don't sit in an Uber car and get an advertising thing.

They're not having to make money like Google and Facebook, which is they provide this free service, quote, unquote, and then they make money with advertising, which has this negative externality. It hurt a lot of people. It led to a lot of our bad externalities. So I'm not completely opposed to advertising.

It's a way to get things out there, but it's, it's, it took over as you don't have to spend much time on it. When you build a producer consumer relationship, you don't need advertising to, to, to get going. And then if you build a three way thing. So anyway, Amazon, yes, also is a producer consumer. I've got people with a product to sell.

I got packages, I will deliver them. And there are people that are buying the packages. So you already got a fair [01:08:00] amount, but you got many of the three way kind of triangles at Amazon and, and they're, they, so they're so called behind on LLMs. But I think that's appropriate. You shouldn't be like it. I think you should take six months to do it better and to do it in the context of a real mechanism.

So I know they're building LLMs that is publicly announced to help the third party seller navigate. I'm trying to sell a new, you know, vacuum cleaner, how much inventory do I need, how much am I going to pay for returns, blah, blah, blah, how all the pricing that's needed there, how do I navigate all that?

Well, LLMs can help you with that. And so that's a great example of building an LLM that, that's just a little LLM in some ways inside of a bigger system, but can be very useful for helping the economy to work. 

hugo: Yeah, and there are a lot of people who are saying Amazon is behind on LLMs and Google's behind on LLMs and this type of stuff.

And I, I think. We're in very early days. It's all in its nascency. And they all make those claims. Don't understand the role of incumbents as well and how they do buy their time. They don't understand 

mike: the economics at all. And they don't understand the idea that you have to have a business [01:09:00] model. 

hugo: Yes, exactly.

mike: So they they've all watched these companies grow up from nothing and the business model always became advertising and maybe subscriptions. And just the negative side of that is very evident. So come on folks, think of a better business model from day one that actually brings wealth into the system in a different way than just this artificial advertising thing, which is very, very disruptive to, you know, human happiness.

hugo: So Mike, I've got one last question. I would have to fire myself if I didn't ask you to perform an active prediction, given you've been working in machine learning and been a hero of so many of us working in machine learning for decades. For so long and I'd like to hear maybe even an optimistic or idealistic vision of where you would Like to see the future of quote unquote intelligent systems and intelligent infrastructure End up 

mike: I think it might take is that my whole discussion with you has been pretty optimistic in tone compared to a lot of my colleagues that either a [01:10:00] way to naively optimistic which I don't think it's a form of optimism this science fiction ever referred to versus my dower Doomer folks who were pessimistic I'm neither I really think we're building things that can and do help people in lots and lots of ways.

Also, I this Notion that, why we have all these stresses and strains, a lot of it comes from the fact that we have 7 billion people on the planet. And that's not going to go away, it's only going to grow. You could naively say, if we got rid of half the people, then life would be easier. And we're not going to have all the climate problems, and so on and so forth.

That's just not what happened. That we're living happily with the fact that each person gets to decide. And responsible people say, okay, that's a given. Then I've got to work with mechanisms that will help cope with that. And I do think a lot of Western people that are optimistical in the sense that they got a lot of wealth.

They're optimistic about keeping that wealth. I don't like that. Or they're optimistic about our technology. We can send it to Africa and it will just. Provide all the medical treatments, it'll provide all the education [01:11:00] because our technology will handle that. That's not optimism, that's naive science fiction.

I'm optimistic that if you do this in a transparent, open way, and scale it and think about the real problems there, that people in Africa will see it and they will take it and do it themselves. That this will be an open technology, that will be built by people in their context, and will aid people like search engines did.

I think mostly search engines were good, and even if it was advertising based, I think it mostly helped people throughout the world. And so my optimism is let's do more things like that but now we have bigger stakes and bigger opportunities but let's do, think about it this way. There is very much a side of me that believes that humans will take a dialogue like this either from the others and will run with it.

And they won't just continue. So I, my pessimism comes out when I get up and read the newspaper in the morning and I talked to, and I see journalists just not getting it at all. And just really the dialogue is so warped and so unfortunate that I, then a little pessimism comes up, we're just, we're going to be [01:12:00] run by these idiots.

And that probably may be true for a little while. They're, I'll leave it at that. I can, even there have optimism, but it's not obvious that I should have it. 

hugo: And of course they have their own skewed incentives as well. I do want to sneak in one final question because our audience is technical people, data scientists, AI practitioners.

Yeah. In terms of the types of systems you think are worth building and what we should be doing, thinking about uncertainty, all of these things, what's one takeaway you would like? Build more 

mike: systems like this thing called United Masters. Build systems where you think about is there a plausible kind of market economy thing that would arise from this piece of technology that I'm building.

At least think about that a little bit, because I think a lot of my colleagues in AI just said, Oh no, my goal is to build a robot that is a humanoid. And that if my robot can go on the stage and dance and sing and act like a human, I've succeeded. Okay, fine, I don't have anything necessarily against that per se, it's just an okay thing to do.

But it's a [01:13:00] trick and it's just such a limited thing to do in your life. Okay. And aspire to something bigger and better. A lot of my robotics colleagues are definitely not in that. They would think about, I'm trying to help with like old people need some help. If they fall down, a robot should be there to help them.

That's all true. And they could even take that a little bit further about, is there an actual system you could build that would, that make human life better and all that. So people do that, but it just, and domain after domain, think that through a little bit. What is my quote unquote business model? Okay.

And don't just think that I'm going to build a piece of technology. And it'll make money. Be a little bit more sober and thoughtful about what you're doing. Yes, for me, the United Masters has been a nice example, because I love music, I love musicians. I want to find a way that they don't just get drips and draps, and that, you know, eventually regulation comes in and makes sure that things get distributed or something.

That's not going to work. And so I want to spend a lot of my time thinking about, well, can I imagine something that could work? And I totally can. It's not that hard. So, that's probably the big message here. [01:14:00] Yes, study your statistics, your economics, and do all that. That's not the real message. The real message is think about it.

Things like markets, don't be afraid of that terminology. Uh, and think about business models, don't be afraid of that terminology. But it's not just so you can make money. And so you can actually make that positive impact you do want to make. And I actually, I'm there, I'm optimistic. I don't think most people just want to make money.

I think they want to actually do things that really help people. 

hugo: I think that's a beautiful note to end on, Mike. And I just want to say thank you, um, for your time. Expertise, but also your generosity and having, having this long conversation as well. 

mike: My pleasure. It's been a fun conversation. I've enjoyed it.

Thank you. We'll go. 

hugo: Thanks so much for listening to high signal brought to you by Delfina. If you enjoyed this episode, don't forget to sign up to our newsletter, follow us on YouTube and share the podcast with your friends. All the links are in the show notes. We'll catch you next [01:15:00] time.