# How AI Is Upending the World of Mathematics

An episode of [Odd Lots](https://invisiblenetwork.ai/podcast/1056200096), 9 Oct 2026, 1 h 00 min.

Audio: https://podtrac.com/pts/redirect.mp3/tracking.swap.fm/track/UVBrz8bN8aM2Xe47PEPu/traffic.omny.fm/d/clips/e73c998e-6e60-432f-8610-ae210140c5b1/8a94442e-5a74-4fa2-8b8d-ae27003a8d6b/f49c2ca3-dfe7-4a67-bc11-b4dd01463f66/audio.mp3?utm_source=Podcast&in_playlist=982f5071-765c-403d-969d-ae27003a8d83

Last month, OpenAI announced that it had produced an AI-generated proof for the Navier-Stokes problem, one of the most famous unsolved questions in mathematics. LLMs used to be bad at counting, but now they are solving math problems that have stumped humans for decades. Meanwhile, at universities, the problem of AI in education continues: Now that LLMs can do a student's homework, teachers are struggling to keep up. It is clear that AI is very quickly changing how math is taught and how it is practiced by professionals. On this episode, Justin Solomon, who is the associate dean for engineering education at MIT, gives us a primer on how mathematicians (both pure and applied) are responding to all the advances in AI. He also explains what exactly the Navier-Stokes problem is and why the OpenAI proof is hard for even the pros to parse, what movies get wrong about how mathematicians do their jobs, and how he's changing his pedagogical approach in the age of AI. See Odd Lots live in Chicago! Tickets on sale here See omnystudio.com/listener for privacy information.

## Transcript

[0:00] Speaker 1: Hey, Odd Lots listeners, the Odd Lots tour continues and
[0:02] Speaker 1: our next stop is in Chicago.
[0:04] Speaker 2: That's right. Joe and I will be at the City
[0:06] Speaker 2: Winery Chicago on October 15th for a live Odd Lots recording.
[0:10] Speaker 2: Tickets are on sale now at Bloomberg.com forward slash Odd Lots.
[0:14] Speaker 1: And of course, a special thank you to Barclays for
[0:17] Speaker 1: supporting Odd Lots Live.
[0:19] Speaker 2: So that's October 15th at City Winery in Chicago. Get
[0:23] Speaker 2: your tickets now.
[0:26] Speaker 1: Bloomberg Audio Studios. Podcasts, radio, news. Hello and welcome to
[0:43] Speaker 1: another episode of the Odd Laws Podcast. I'm Joe Weisenthal.
[0:47] Speaker 2: And I'm Tracy Allaway.
[0:48] Speaker 1: So Tracy, with AI, you know, obviously one of the
[0:51] Speaker 1: big concerns is like job loss. What are we going to,
[0:55] Speaker 1: in the future, what are humans going to be better
[0:58] Speaker 1: at and What.
[0:59] Speaker 2: Are we going to do all day?
[1:00] Speaker 1: What are we going to do all day, et cetera?
[1:02] Speaker 1: And we really don't know the answer, I think, to
[1:04] Speaker 1: any of these questions yet. But I am aware, obviously,
[1:09] Speaker 1: the models are getting very good at math, and at
[1:11] Speaker 1: least some mathematicians perceive, you know, they're in a state
[1:15] Speaker 1: of existential angst about this, and certainly crisis, perhaps, some
[1:20] Speaker 1: feel about, like, well, what are we doing here? These
[1:23] Speaker 1: problems that I devoted my life to solving and the
[1:27] Speaker 1: model solves it in a weekend. Is this what's potentially
[1:31] Speaker 1: coming for many of us?
[1:32] Speaker 2: Yeah. Math is an interesting one because you can tell
[1:35] Speaker 2: that a lot of the AI companies themselves have an
[1:38] Speaker 2: interest in math. They see math as a sort of
[1:40] Speaker 2: holy grail for, I guess, proving that the models are
[1:43] Speaker 2: able to reason. It's the thing they need to do
[1:46] Speaker 2: and able to say that they've achieved AGI, I think,
[1:49] Speaker 2: is the way it's couched. And it also makes good headlines.
[1:53] Speaker 2: So even though I have no idea what the Navier-Stokes
[1:57] Speaker 2: problem actually is. I don't know even if I'm pronouncing
[1:59] Speaker 2: that correctly. Is it Navier or Navier? Okay, Navier-Stokes. I
[2:03] Speaker 2: see the headline and I'm like, ooh, a model has
[2:06] Speaker 2: solved the Navier-Stokes problem, as if I know what that
[2:10] Speaker 2: actually means and the significance for the mathematics profession.
[2:13] Speaker 1: It's also funny because I downloaded the paper and it's
[2:17] Speaker 1: like one paragraph and I have no idea. I can't
[2:20] Speaker 1: even begin to parse the output. Another fascinating aspect of
[2:26] Speaker 1: this is just, like, it would not have been intuitive
[2:29] Speaker 1: to me, say, two years ago or three years ago,
[2:33] Speaker 1: that a language model would be so good at mathematics.
[2:37] Speaker 1: In fact, at one point, they weren't very good at
[2:39] Speaker 1: math at all. Like, go back to, like, I don't know,
[2:41] Speaker 1: GPT-3 or GPT-3.5.
[2:44] Speaker 2: If you asked them to count, like, the number of
[2:46] Speaker 2: letters in a word or something, they would get it wrong.
[2:49] Speaker 3: Yeah.
[2:50] Speaker 1: And weirdly, they still, like, struggle with certain things like that.
[2:54] Speaker 1: But it's not obvious to me, like, okay, you scan
[2:58] Speaker 1: all of the text on text, you know, language, et cetera.
[3:02] Speaker 1: And yet somehow that generalizes into this extraordinary math capability.
[3:09] Speaker 1: That too is very surprising.
[3:10] Speaker 3: Yeah.
[3:10] Speaker 2: In this conversation, can I just say before we start,
[3:13] Speaker 2: I would also just like to learn what advanced mathematics
[3:16] Speaker 2: actually is. Because everything I know about it, which is nothing,
[3:20] Speaker 2: comes from like watching Good Will Hunting and just seeing,
[3:23] Speaker 2: you know, a bunch of formulas on a chalkboard and
[3:25] Speaker 2: people get really excited when they start crossing off certain
[3:28] Speaker 2: parentheses and stuff like that. I have no idea what
[3:31] Speaker 2: it is. I have no idea how it differs from
[3:33] Speaker 2: pure maths of like two plus two equals four. And
[3:37] Speaker 2: so I would just like to come away in this
[3:39] Speaker 2: conversation with a better understanding of what it is overall
[3:42] Speaker 2: and how AI actually fits into it.
[3:44] Speaker 1: Absolutely. So like I was thinking along the same lines, like, okay,
[3:48] Speaker 1: for like someone who like devotes their life to like
[3:51] Speaker 1: advanced theoretical math and, you know, other also like things
[3:56] Speaker 1: like advanced theoretical physics and, that don't necessarily have like
[4:00] Speaker 1: obvious like industrial applications, etc. Who is the type of person?
[4:05] Speaker 1: What are they hoping to achieve? Why is there still
[4:08] Speaker 1: interest in some of these things that perhaps the sort
[4:12] Speaker 1: of applicability is not necessarily so obvious? But then like obviously,
[4:19] Speaker 1: you know, like what is the role of the mathematician here?
[4:22] Speaker 3: At this point.
[4:23] Speaker 1: And then actually specifically it's like, okay, across universe. This
[4:27] Speaker 1: is actually the thing I'm very curious about, which is
[4:30] Speaker 1: my son studies math, right? Both my kids study math
[4:33] Speaker 1: and like.
[4:33] Speaker 2: Hopefully not theoretical math just yet.
[4:36] Speaker 1: Not yet.
[4:37] Speaker 3: How old is he?
[4:38] Speaker 1: My daughter's 10. My son is seven. They take math,
[4:42] Speaker 1: et cetera. But like setting aside like what the future holds,
[4:46] Speaker 1: I'm very interested in just the present. Because it's like
[4:48] Speaker 1: you have all these math departments at universities all around
[4:52] Speaker 1: the country and like maybe some are excited about AI,
[4:54] Speaker 1: et cetera. But like, what do you do today in
[4:58] Speaker 1: terms of like rethinking pedagogies? Because it must be, or
[5:02] Speaker 1: I would assume it's pretty different in 2026 than like
[5:06] Speaker 1: those same departments in 2021 or something like that.
[5:10] Speaker 2: Absolutely.
[5:10] Speaker 1: And like, how are professors, teachers, et cetera, thinking about
[5:16] Speaker 1: just the sort of the process of teaching math right now.
[5:20] Speaker 1: It's just like, just sort of like very straightforwardly, how
[5:22] Speaker 1: is it affecting their jobs? I used to be really,
[5:25] Speaker 1: really good at sort of mental math, like times tables
[5:27] Speaker 1: and stuff like that.
[5:28] Speaker 2: This is a question I have too. If you are
[5:30] Speaker 2: an advanced mathematician doing theoretical maths, how often do you
[5:34] Speaker 2: actually use a calculator?
[5:36] Speaker 1: I'm curious about that.
[5:37] Speaker 2: That's going to be my first question.
[5:38] Speaker 1: Actually, Jensen Wong, I don't know if you read the
[5:41] Speaker 1: transcript of his interview with Ezra Klein a couple of
[5:45] Speaker 1: weeks ago.
[5:45] Speaker 3: Mm-hmm.
[5:46] Speaker 1: But he said something. It's like, oh, maybe it's just not.
[5:49] Speaker 1: It's a little kind of weird interview. He said something like, well,
[5:51] Speaker 1: maybe like knowing your times tables isn't going to be
[5:54] Speaker 1: an important skill in the future.
[5:55] Speaker 2: Or your address.
[5:56] Speaker 1: Yeah, which is like kind of weird. Like, I still
[5:59] Speaker 1: want my kids to.
[6:01] Speaker 2: No, but I think it raises a very important question,
[6:05] Speaker 2: which is you can go to a model and have
[6:08] Speaker 2: them verify a proof or a theorem or something like that.
[6:12] Speaker 2: But if you don't understand how you actually got there
[6:16] Speaker 2: and how the model is getting to its verification, then
[6:19] Speaker 2: what are we actually learning here?
[6:21] Speaker 1: Right. And then there's the question of like. people learn
[6:24] Speaker 1: a lot in the process, right? So it's like even
[6:27] Speaker 1: presumably in just the people work on some of these
[6:30] Speaker 1: problems for years and years, and maybe they never arrive
[6:33] Speaker 1: at any proof, but intuitively you would think there is something,
[6:37] Speaker 1: maybe it's exercising a muscle or something, or sort of
[6:41] Speaker 1: other benefits to just the process of learning. How's that change? Anyway,
[6:45] Speaker 1: I have a lot of questions about this and we
[6:47] Speaker 1: really do have the perfect guest. We're going to be
[6:49] Speaker 1: speaking with Justin Solomon. He's Associate Dean of Engineering Education
[6:54] Speaker 1: at MIT and very interested and involved in math education
[6:58] Speaker 1: and thinking about the intersection of math and AI. Justin,
[7:02] Speaker 1: thank you so much for coming on OddLots.
[7:03] Speaker 3: Thanks for having me.
[7:05] Speaker 1: I mean, Tracy sort of already suggested, but let's set
[7:08] Speaker 1: aside AI for a second. What is it that people
[7:12] Speaker 1: who end up going into sort of like advanced math
[7:15] Speaker 1: or advanced theoretical math, like who are these people?
[7:19] Speaker 2: Why?
[7:20] Speaker 1: Like, what are they hoping to do?
[7:22] Speaker 3: Yeah, it's true. I feel like the typical caricature that
[7:25] Speaker 3: you see is maybe this character on, what, Good Will Hunting,
[7:28] Speaker 3: which incidentally, they're always writing math on Windows, which is
[7:31] Speaker 3: very impractical. Good visuals, though. Yeah, that's true. It's great
[7:35] Speaker 3: for a camera. But that aside, I think it's an
[7:37] Speaker 3: intensely personal thing. So to be clear, my personal career,
[7:41] Speaker 3: like I'm a professor of engineering, is in an area
[7:43] Speaker 3: of math called applied math, which sometimes is applied and
[7:46] Speaker 3: sometimes not. That's a discussion for another day. But essentially,
[7:48] Speaker 3: some people go into math because they want to see
[7:50] Speaker 3: it translated into applications or to bring fundamental insight into
[7:54] Speaker 3: the kind of things we do every day, whether it's
[7:56] Speaker 3: predicting markets or physics, what have you. Other people in
[7:59] Speaker 3: areas of math, sometimes you'll call them pure math. I
[8:02] Speaker 3: don't love that terminology, like pure versus applied. But in
[8:05] Speaker 3: other areas of math, it's almost just for the joy
[8:07] Speaker 3: of the game or understanding the universe around you. All
[8:10] Speaker 3: of these are totally legitimate reasons and sort of.
[8:13] Speaker 1: intellectual pursuit for the sake of intellectual pursuit.
[8:17] Speaker 3: That's right. But along the way, I think you develop
[8:20] Speaker 3: a lot of really important reasoning skills and skills that
[8:22] Speaker 3: support work in all kinds of different areas.
[8:25] Speaker 2: And how often do advanced mathematicians, whether they're in, I guess,
[8:29] Speaker 2: applied maths or purely theoretical maths, how often do they
[8:33] Speaker 2: use a calculator?
[8:34] Speaker 3: Very often. Okay. Yeah, no. But my impression, again.
[8:37] Speaker 2: Is everyone is doing this by hand on chalkboards and windows.
[8:41] Speaker 3: I mean, there's like lots of Greek symbols and letters
[8:43] Speaker 3: and variables and things. But no, I'm terrible at arithmetic.
[8:46] Speaker 3: I've got a calculator out all the time. Oh, yeah.
[8:49] Speaker 2: This gives me hope, actually, that maybe I could understand
[8:53] Speaker 2: some of this, even though I'm pretty bad at arithmetic itself.
[8:56] Speaker 3: Oh, absolutely.
[8:58] Speaker 2: What's the difference between a math problem then and a
[9:00] Speaker 2: math proof or a theorem? These sort of big picture
[9:03] Speaker 2: things that we're talking about AI models actually doing now.
[9:06] Speaker 3: That's a great question. And it's really important to dig
[9:08] Speaker 3: into exactly what the models are doing and not doing
[9:11] Speaker 3: and who deserves credit, by the way. But essentially, in mathematics,
[9:14] Speaker 3: it's all about verifying, in some sense, verifying true facts
[9:18] Speaker 3: is one part of the puzzle. But it's also all
[9:21] Speaker 3: about figuring out what are the interesting problems to study
[9:24] Speaker 3: and what insight they bring to the broader world, right?
[9:27] Speaker 3: So in some areas of mathematics, there are all kinds
[9:30] Speaker 3: of different questions you could ask. And some of them
[9:31] Speaker 3: are super uninteresting, but you could ask a computer to
[9:33] Speaker 3: prove them. And others maybe tell you something about a
[9:36] Speaker 3: physical system or about a system of equations or something
[9:39] Speaker 3: like that, that bring broader understanding. And so this exercise
[9:42] Speaker 3: of proofwriting is all about sort of writing a mathematically
[9:47] Speaker 3: convincing argument that tells mathematicians. So there's a human aspect,
[9:51] Speaker 3: which is an important one. that a certain fact is
[9:53] Speaker 3: actually maybe reality, right? It aligns with logic. There's not
[9:56] Speaker 3: some flaw hiding in there somewhere.
[9:59] Speaker 2: And where do you ideas for a particular mathematical proof
[10:03] Speaker 2: or theory come from? Because again, I'm going to come
[10:05] Speaker 2: at this from, I guess, the movie angle, but I
[10:07] Speaker 2: think about A Beautiful Mind, right? And the famous bar
[10:10] Speaker 2: scene and it was John Nash, right? That was his name.
[10:14] Speaker 2: And he has this like big realization while he's at
[10:17] Speaker 2: the bar with these women and it ends up turning
[10:20] Speaker 2: into a famous mathematical theory. Is it up, like where
[10:24] Speaker 2: did the ideas actually come from for the rules that
[10:26] Speaker 2: you are trying to prove in maths or these big
[10:29] Speaker 2: picture ideas?
[10:30] Speaker 3: That's a great question. So sometimes in mathematics, there are
[10:34] Speaker 3: some really natural questions out there in the universe that
[10:36] Speaker 3: people have been asking for a long time. So for example,
[10:38] Speaker 3: these days in the news, we hear about some of
[10:40] Speaker 3: these really high profile math problems that have been around forever.
[10:43] Speaker 3: There are other types of mathematicians that are more theory builders,
[10:46] Speaker 3: where they're the ones asking the questions rather than answering
[10:48] Speaker 3: someone else's questions. There are definitely different approaches that are
[10:51] Speaker 3: out there. Now, when it comes to where these questions
[10:54] Speaker 3: are coming from, some of them are inspired purely mathematically,
[10:58] Speaker 3: and other ones maybe come from a very practical setting.
[11:00] Speaker 3: Maybe somebody observed some phenomenon out there in the real world,
[11:03] Speaker 3: and they're trying to explain it with the smallest model possible. Now,
[11:07] Speaker 3: one thing I should clarify is that math is a
[11:09] Speaker 3: really social exercise. I think that's something that people don't
[11:12] Speaker 3: necessarily understand, especially because these movies really play into this
[11:16] Speaker 3: almost great man theory of mathematics. But that's just not
[11:18] Speaker 3: how it works. There's this huge community of people that
[11:21] Speaker 3: are working on important problems. They're communicating with one another, collaborating.
[11:25] Speaker 3: And I would say, for example, from my own research,
[11:27] Speaker 3: almost everything that our team has worked on comes as
[11:30] Speaker 3: a team effort. We're all sitting in the same room.
[11:32] Speaker 3: We're at the blackboard writing computer models, what have you.
[11:35] Speaker 3: And that's where the insight is coming from. So it's
[11:37] Speaker 3: not just sitting in your ivory tower. It's being out
[11:39] Speaker 3: there in the world, observing things happening and talking to
[11:42] Speaker 3: your colleagues.
[11:43] Speaker 1: Before we get more into the sort of AI question,
[11:46] Speaker 1: I've actually been curious, what's applied math? What is the
[11:50] Speaker 1: difference between applied math and pure math?
[11:52] Speaker 3: Yeah, it's a great question. So there's sort of a
[11:55] Speaker 3: divide in the math world, but I think it's fuzzier
[11:58] Speaker 3: than maybe it sounds when I describe it here. So roughly,
[12:01] Speaker 3: there's an area of mathematics that you might call applied math.
[12:04] Speaker 3: where the goal is to either derive theoretical results or
[12:07] Speaker 3: even come up with computer algorithms that explain phenomena out
[12:10] Speaker 3: there in the real world. So, for example, in applied math,
[12:13] Speaker 3: maybe you work on algorithms that simulate the weather, and
[12:16] Speaker 3: behind the scenes is a really complex mathematical model, and
[12:19] Speaker 3: it's all about making sure that your model is correct
[12:21] Speaker 3: and efficient and well-behaved, all that good stuff. In pure mathematics,
[12:25] Speaker 3: sometimes the motivation is a little bit less attached to
[12:28] Speaker 3: a particular application, and more just attached to the math
[12:31] Speaker 3: problem itself.
[12:32] Speaker 1: So something like Fermat's Last Theorem or something like that,
[12:35] Speaker 1: where he's like, okay, I forget what it is exactly,
[12:39] Speaker 1: but that would not be applied math.
[12:41] Speaker 3: Right, so Fermat's Last Theorem, and many things I think
[12:43] Speaker 3: in number theory fall into that category, but also many
[12:46] Speaker 3: times these things bleed into one another. So for example,
[12:48] Speaker 3: a lot of modern developments in cryptography came from areas
[12:52] Speaker 3: of mathematics that we used to think of as extremely
[12:54] Speaker 3: pure and abstract.
[12:56] Speaker 1: Oh, interesting.
[12:57] Speaker 2: This is also Good Will Hunting, by the way.
[12:59] Speaker 3: It all goes back to that.
[13:00] Speaker 2: That's right.
[13:01] Speaker 3: That's right.
[13:02] Speaker 2: You worked at Pixar before.
[13:03] Speaker 3: Many years ago.
[13:04] Speaker 2: Was that applied maths working at Pixar?
[13:06] Speaker 3: Oh, I love that question. The answer is absolutely yes. Okay. Yeah.
[13:09] Speaker 3: So the movie studios are a great example of a
[13:12] Speaker 3: place where I would say state of the art and
[13:14] Speaker 3: certain types of mathematics are taking place. So when you
[13:17] Speaker 3: watch a typical movie with different movie effects, there's cloth,
[13:21] Speaker 3: there's smoke, there's all kinds of fluids, all rigid bodies
[13:25] Speaker 3: flying around. All the dynamics of those are governed by
[13:28] Speaker 3: things called partial differential equations. And coming up with the
[13:31] Speaker 3: right algorithms for simulating those, and by the way, making
[13:34] Speaker 3: them so well behaved that an artist doesn't have to
[13:36] Speaker 3: think about math, is a really challenging mathematical problem. So
[13:40] Speaker 3: I typically tell my students that if you're looking for
[13:43] Speaker 3: really challenging math problems, you don't have to make them up.
[13:45] Speaker 3: They're out there in the real world. And that's a
[13:46] Speaker 3: nice example.
[14:03] Speaker 1: What's Navier-Stokes?
[14:04] Speaker 3: Okay. So Navier-Stokes has been in the news a lot lately.
[14:08] Speaker 3: And it's been great for those of us with a
[14:10] Speaker 3: little bit of background in differential equations.
[14:14] Speaker 1: Getting lots of media inquiries.
[14:16] Speaker 3: That's exactly right. Yeah, we have our 50 minutes of
[14:18] Speaker 3: fame here. It'll die fast, but we'll enjoy it for now. Right.
[14:22] Speaker 3: So the Navier-Stokes problem is one that comes from the
[14:24] Speaker 3: world of fluid dynamics. So the Navier-Stokes equations were this model.
[14:27] Speaker 3: They go back quite a long time. that explain the
[14:30] Speaker 3: motion of, for example, liquid in a cup. So like
[14:32] Speaker 3: both of you guys have coffee cups here on the table.
[14:34] Speaker 3: And if you like shake that coffee cup up, you
[14:36] Speaker 3: know the initial velocity, you know what's inside of the cup.
[14:39] Speaker 3: Then the Navier-Stokes equations are trying to predict like where
[14:43] Speaker 3: the fluid moves over time, right? Like where is the velocity, vorticity,
[14:47] Speaker 3: all these different things. Now, the Navier-Stokes equations are really complicated.
[14:51] Speaker 3: I mean, when you look at your latte, when you
[14:53] Speaker 3: stir it in the morning and you see the milk
[14:55] Speaker 3: making these cool shapes and so on, there's all kinds
[14:57] Speaker 3: of complex behavior that can happen. And so the Navier-Stokes equations,
[15:02] Speaker 3: although they're easy to write down, they're very hard to
[15:04] Speaker 3: say a whole lot about. And so there was this
[15:07] Speaker 3: big open math problem, which I think is what you
[15:09] Speaker 3: guys are referring to because it's been in the news,
[15:11] Speaker 3: which involves existence and uniqueness of solutions. So the basic
[15:14] Speaker 3: idea here, I think actually my undergrad differential equation professor
[15:18] Speaker 3: years ago explained it to me in kind of a
[15:20] Speaker 3: funny way. He sort of said, you know, all you
[15:22] Speaker 3: got to do to make a million dollars is prove
[15:24] Speaker 3: that water doesn't explode. And that's roughly right. I mean,
[15:27] Speaker 3: with a lot of asterisks. Essentially, what we didn't know
[15:30] Speaker 3: is if you know this initial configuration of your fluid
[15:33] Speaker 3: and you let it kind of go under its dynamics
[15:35] Speaker 3: over time, then whether it exists for all time and
[15:39] Speaker 3: doesn't have some really crazy turbulent behavior, or if, for example,
[15:43] Speaker 3: maybe it blows up or it develops a singularity or
[15:45] Speaker 3: something like that. So this was kind of a yes
[15:47] Speaker 3: or no question, like prove or disprove. And I guess
[15:50] Speaker 3: the recent news was actually a counterexample. So in other words,
[15:53] Speaker 3: it showed in some finite amount of time that with
[15:58] Speaker 3: an asterisk, which is with a very crazy spoon, if
[16:00] Speaker 3: you stir your fluid just the right way, you really
[16:03] Speaker 3: can create this totally singular and frankly, non-physical behavior. Interesting.
[16:07] Speaker 1: Proving in math is like one of these things. Like
[16:12] Speaker 1: I think I, the formal proof that two plus two
[16:15] Speaker 1: equals four, it's like a nine step thing, isn't it?
[16:19] Speaker 3: Right?
[16:19] Speaker 1: Like if you are like, sometimes eight, sometimes eight, but like,
[16:23] Speaker 1: Can you explain what a proof is?
[16:26] Speaker 3: Sure.
[16:27] Speaker 1: I know 2 plus 2 equals 4. It seems obvious.
[16:30] Speaker 1: But explain what is a proof such that to say
[16:35] Speaker 1: that to prove that 2 plus 2 equals 4 is
[16:38] Speaker 1: something that requires eight steps.
[16:40] Speaker 3: Right. This idea of proof writing, I think, is actually
[16:43] Speaker 3: you can think of it as almost the result of
[16:45] Speaker 3: an identity crisis among mathematicians maybe 100, 200 years ago.
[16:49] Speaker 3: where it used to be a lot of math was
[16:50] Speaker 3: drawing pictures and kind of working in the same fashion
[16:53] Speaker 3: that you're talking about, kind of intuitively. But at some point,
[16:56] Speaker 3: people realized that they wanted to make sure that the
[16:57] Speaker 3: field had really strong foundations. And so mathematicians will agree
[17:01] Speaker 3: on a certain set of maybe logical principles or rules,
[17:05] Speaker 3: I would say that's more the domain of philosophy, precisely
[17:07] Speaker 3: what those should be. And then the idea of proofreading
[17:10] Speaker 3: is to build up from these kind of basically, we
[17:13] Speaker 3: agree that these five facts are true, to a much
[17:16] Speaker 3: more complicated fact by combining them in different and creative ways.
[17:19] Speaker 3: And so theoretical mathematics oftentimes is all about, I don't
[17:23] Speaker 3: want to just say mixing and matching different facts, but
[17:25] Speaker 3: coming up with smart ways to justify mathematical ideas built
[17:29] Speaker 3: on very simple principles and steps from logic.
[17:32] Speaker 2: So when it comes to AI currently, is the role
[17:35] Speaker 2: more in, I guess, proof checking versus proof generation or
[17:40] Speaker 2: idea generation? Like what are the models actually doing.
[17:43] Speaker 3: At the moment? Well, they're sort of doing it all
[17:45] Speaker 3: and they're sort of doing nothing. So, right. The current
[17:48] Speaker 3: AI models in terms of the types of mathematics they're doing,
[17:52] Speaker 3: I don't think it's quite right to say that it's
[17:53] Speaker 3: just proof checking. And in fact, actually, they're quite bad
[17:56] Speaker 3: at proof checking. And one of the revolutions in, I
[17:59] Speaker 3: would say, the last year or so is the realization
[18:02] Speaker 3: that actually what you have to do is to provide
[18:04] Speaker 3: your model, like your large language model, with a second
[18:07] Speaker 3: module that can check the AI's proof. And so, for example,
[18:12] Speaker 3: there's a programming language called Lean, which is basically just
[18:14] Speaker 3: a sort of codified version of these mathematical rules. And
[18:18] Speaker 3: these days what people will do is they'll put in
[18:20] Speaker 3: their conjecture, so like the thing that they want to
[18:22] Speaker 3: prove or disprove, and then ask the AI system to
[18:24] Speaker 3: generate a proof. But then they'll say rather than just
[18:26] Speaker 3: writing it in human-readable language, which is the substrate that
[18:29] Speaker 3: mathematicians use to communicate with each other, maybe in addition
[18:32] Speaker 3: to that, they'll actually produce this lean code so that
[18:36] Speaker 3: there's a separate piece of software out there that can
[18:38] Speaker 3: check that this proof truly is correct. So actually the
[18:41] Speaker 3: correctness part of it is usually deferred to another piece
[18:44] Speaker 3: of software, which isn't AI, but somehow because of the
[18:47] Speaker 3: interaction between the AI tool and this proof checker, that's
[18:51] Speaker 3: what's created a lot of progress that we can believe
[18:52] Speaker 3: in a little bit more than we did in the past,
[18:54] Speaker 3: if that makes sense.
[18:56] Speaker 1: I have heard of, only because, you know, I've been
[18:59] Speaker 1: reading the news and I've heard of this thing called Lean.
[19:02] Speaker 3: Yes.
[19:02] Speaker 1: And it sort of seems extraordinary to me. Can you
[19:06] Speaker 1: explain it a little bit? It seems extraordinary to me
[19:09] Speaker 1: that there is this, I don't know, is it software
[19:11] Speaker 1: or this framework such that these extraordinarily complex problems or solutions,
[19:17] Speaker 1: I guess is the answer, can be dropped into this
[19:21] Speaker 1: and it's capable of like, Yes, this is available.
[19:24] Speaker 2: But it's also kind of crazy that you need basically
[19:27] Speaker 2: a translator for it, right?
[19:29] Speaker 3: Yeah.
[19:29] Speaker 1: Can you explain a little bit like how Lean works, perhaps?
[19:32] Speaker 3: I can explain a little bit. To be clear, I'm
[19:34] Speaker 3: not an expert in this particular corner. But essentially, as
[19:38] Speaker 3: we all know, things that come out of AI are unreliable.
[19:40] Speaker 2: Yeah.
[19:41] Speaker 3: All of your listeners should know this. We all know this.
[19:44] Speaker 3: And that goes for mathematics as well. And math we
[19:46] Speaker 3: think of as this area that is extremely reliable, right?
[19:49] Speaker 3: Everything should be true or false. Now, there are gray areas,
[19:51] Speaker 3: but certainly we try to avoid that. So this programming
[19:54] Speaker 3: language lean, what it does is it has a few
[19:56] Speaker 3: simple maybe axioms that are built in. So math things
[19:59] Speaker 3: that we sort of agree are the starting points for
[20:02] Speaker 3: writing proofs, like the steps of basic logic. And then
[20:05] Speaker 3: what happens is that you write a theorem in this
[20:08] Speaker 3: lean language. So rather than in English, you write it
[20:10] Speaker 3: in this programming language. And then it's kind of cleverly
[20:13] Speaker 3: designed so that once the software checks that that theorem
[20:17] Speaker 3: is true, it can get added to the library of
[20:19] Speaker 3: sort of true facts, and then you can build up.
[20:22] Speaker 3: So you make this more and more complex edifice out
[20:24] Speaker 3: of these simple steps. And it's been really impressive what
[20:26] Speaker 3: that can do. I mean, some of the modern proofs
[20:28] Speaker 3: that you're seeing, especially the AI-generated ones, are like hundreds
[20:31] Speaker 3: of thousands of lines of lean code. I don't think
[20:33] Speaker 3: a human could check it. It also, by the way,
[20:35] Speaker 3: means that we're trusting that someone wrote That software, frankly,
[20:39] Speaker 3: which is a philosophical question. But, you know, if you
[20:42] Speaker 3: believe that Lean is good at checking proofs, then it
[20:44] Speaker 3: provides a lot of certainty that the things that the
[20:46] Speaker 3: AIs are writing down are actually correct.
[20:49] Speaker 2: Devil's advocate question. But why is it important that we
[20:53] Speaker 2: need to understand the actual problem and the proof? Because
[20:57] Speaker 2: if I'm an engineer and I don't know, some model
[21:01] Speaker 2: somewhere comes up with this really new, interesting mathematical theory
[21:05] Speaker 2: and I can apply it to my day to day work.
[21:07] Speaker 3: Yes.
[21:08] Speaker 2: Is that not enough that like I can use it?
[21:10] Speaker 2: It seems to work for the most part. Maybe it's
[21:13] Speaker 2: been rigorously tested in a physical environment or whatever. Why
[21:17] Speaker 2: do I need to understand everything that's gone into it?
[21:19] Speaker 3: Oh, that's a fantastic question. So let's see. My research
[21:23] Speaker 3: group works a lot on simulation algorithms. So like I
[21:26] Speaker 3: have some physical system and I develop a technique for
[21:28] Speaker 3: predicting what it will look like in the future. And
[21:31] Speaker 3: we spend a lot of time writing theoretical results to
[21:33] Speaker 3: verify that our algorithms are correct. They behave the way
[21:36] Speaker 3: we expect and so on. And that's absolutely critical because
[21:39] Speaker 3: without that, you really can't trust what your software is producing.
[21:42] Speaker 3: So for example, if I'm trying to say yes or no,
[21:45] Speaker 3: is this building going to collapse during an earthquake?
[21:47] Speaker 2: Seems important.
[21:48] Speaker 3: That's right. And all of that is built on mathematical models,
[21:51] Speaker 3: techniques like finite element modeling and so on. They're not
[21:54] Speaker 3: too exciting outside of my universe, but you're all trusting
[21:57] Speaker 3: these things to do really important engineering calculations and things
[22:01] Speaker 3: that sure, like one way to justify your software is
[22:04] Speaker 3: that your building hasn't collapsed yet. but that's not a
[22:06] Speaker 3: whole lot of, you know, I hope I can trust
[22:09] Speaker 3: a little bit better than that.
[22:10] Speaker 1: You know, my dad was a physics professor and I
[22:13] Speaker 1: have memories of watching a video of the famous Tacoma
[22:20] Speaker 1: Narrows bridge collapse. You know, that video, you know, like
[22:25] Speaker 1: that was like for him, like this, like, you know, I,
[22:28] Speaker 1: that was, I've watched it several times in my household.
[22:31] Speaker 3: Our engineering students have to take a lot of ethics
[22:33] Speaker 3: courses that are pretty fraught that way.
[22:36] Speaker 1: Oh, interesting. So why don't you give us, I mean,
[22:40] Speaker 1: there's a couple avenues I want to explore, including just
[22:42] Speaker 1: the process of teaching and how it's different in 2026
[22:47] Speaker 1: versus 2016, given the new technology. But maybe give us
[22:52] Speaker 1: a lay of the land for the math community. Because
[22:55] Speaker 1: I know there are some people in despair.
[22:59] Speaker 3: I believe.
[23:00] Speaker 1: We're very anxious about what this means for the future
[23:03] Speaker 1: of math as an intellectual discipline, etc. And then others
[23:08] Speaker 1: seem to be very enthusiastic about the opportunity of human
[23:12] Speaker 1: and AI exploring new ground. But what are the conversations
[23:18] Speaker 1: that you're in these days with fellow mathematicians?
[23:21] Speaker 3: Oh my gosh, things are changing by the day. And really,
[23:24] Speaker 3: it's a Huge collection of individuals. Every single person has
[23:27] Speaker 3: their own opinion here. So I can tell you my
[23:29] Speaker 3: view of the world. And if you talk to somebody else,
[23:31] Speaker 3: I'm sure they'll have a very different one. Math is
[23:34] Speaker 3: an academic discipline. It also shows up in a lot
[23:36] Speaker 3: of applications, as we mentioned. But every aspect of the
[23:39] Speaker 3: work is changing from how people produce mathematics and choose
[23:43] Speaker 3: problems to work on to how mathematics is sort of
[23:48] Speaker 3: measured in terms of productivity, right? When we talk about
[23:51] Speaker 3: hiring faculty, for example, like what are the standards to
[23:54] Speaker 3: evaluate a person's body of work and what did the
[23:56] Speaker 3: human contribute, I think are really critical questions to like,
[23:59] Speaker 3: why do this exercise at all? Maybe leave this to
[24:01] Speaker 3: a computer. There's all kinds of identity crises that are happening. Now,
[24:06] Speaker 3: among mathematicians, there are people that are maybe theory builders
[24:09] Speaker 3: and there are people that are like theorem provers. The
[24:11] Speaker 3: theorem provers, I think, are especially under threat at the
[24:13] Speaker 3: current moment where like maybe you've been working on one
[24:16] Speaker 3: problem and it's well-defined and you've known about it for,
[24:19] Speaker 3: you know, 50 years and it's this crowning question that
[24:22] Speaker 3: you want to answer about the universe. And these AI
[24:24] Speaker 3: tools are sort of at least good at sort of
[24:26] Speaker 3: cherry picking the ones that are somehow close to being
[24:28] Speaker 3: proved and taking that last mile in some cases. And
[24:32] Speaker 3: other people disagree with that characterization.
[24:34] Speaker 1: What do you mean by when you say they disagree
[24:37] Speaker 1: with it?
[24:37] Speaker 3: Like exactly how interesting, for example, a mathematical proof is,
[24:41] Speaker 3: is a very human judgment, as is what problems you
[24:43] Speaker 3: should or should not actually go about working on in
[24:46] Speaker 3: the real world. So There are a lot of true facts.
[24:48] Speaker 3: It doesn't mean we have to write them all down, essentially.
[24:50] Speaker 3: And because of that, this is definitely a human exercise, right?
[24:54] Speaker 3: To a large extent, mathematical research is about humans communicating
[24:57] Speaker 3: to other humans, right? It's not just about verifying true
[24:59] Speaker 3: things about the universe, but bringing insight and helping people
[25:03] Speaker 3: understand systems and so on. So I think you mentioned
[25:06] Speaker 3: an identity crisis. I think what's happened is, and this
[25:08] Speaker 3: is reflected, there's a really nice write-up by Terence Tao,
[25:11] Speaker 3: who's a famous mathematician, that kind of talks about how
[25:14] Speaker 3: there were a lot of different sort of signs of
[25:17] Speaker 3: mathematical progress that used to be correlated with one another, right?
[25:20] Speaker 3: So for example, if I wanted to judge whether a
[25:24] Speaker 3: mathematician has quality research or output, maybe I look at
[25:27] Speaker 3: where they're publishing, the number of things that they're writing,
[25:30] Speaker 3: the interest of their result, whether they're earning these prizes
[25:32] Speaker 3: and so on. And now these things are a bit
[25:35] Speaker 3: at odds. So for example, the AI companies have figured out,
[25:37] Speaker 3: as you guys pointed out, that proving some big theoretical
[25:41] Speaker 3: result maybe is an interesting press release, right? And now
[25:45] Speaker 3: there's an economic component.
[25:46] Speaker 1: Knowing how to do the right prompt, right?
[25:48] Speaker 3: That's right. And then there's a big question, which is, okay, so,
[25:51] Speaker 3: you know, at least in the sort of cartoon version
[25:53] Speaker 3: of a mathematician, you sit in a room, you think
[25:55] Speaker 3: about a really hard problem, you write down a proof
[25:57] Speaker 3: that's subtle and challenging. Now, as you say, maybe you're
[26:00] Speaker 3: the guy that wrote down the prompt. If the prompt,
[26:02] Speaker 3: for example, were prove or disprove Navier-Stokes equation, did that
[26:06] Speaker 3: guy really bring a whole lot of insight to that
[26:08] Speaker 3: problem is an interesting question and one that I think
[26:10] Speaker 3: we're really grappling with today.
[26:28] Speaker 2: I want to ask more questions about credit, but just
[26:31] Speaker 2: on the models themselves, does sycophancy exist when it comes
[26:35] Speaker 2: to math prompts the same way it exists for other,
[26:39] Speaker 2: I guess, language-based queries? Does the model go like, oh,
[26:43] Speaker 2: what an interesting problem. You're on the right track.
[26:47] Speaker 3: You're on the right track. Judging from my email box,
[26:50] Speaker 3: I think the answer is yes. I'll tell you a
[26:52] Speaker 3: funny anecdote. Right, so all the time I get emails
[26:55] Speaker 3: from people that are looking for an expert to maybe verify,
[26:57] Speaker 3: you know. Oh, yeah. And what I've noticed lately is
[27:00] Speaker 3: I get a lot of emails from outside folks that
[27:02] Speaker 3: sort of, they've been playing with their AI system, they've
[27:04] Speaker 3: proven a new result, and the AI told them it
[27:06] Speaker 3: was really important and they need an MIT professor to validate,
[27:09] Speaker 3: which is absolutely fascinating. In fact, for fun, I played
[27:11] Speaker 3: with some of these tools and put in a few
[27:13] Speaker 3: open problems in my own domain. Interestingly, they did not
[27:16] Speaker 3: get solved, so apparently my corner of math is safe
[27:19] Speaker 3: at the moment. But... the first thing that it did,
[27:22] Speaker 3: it proved some, from my perspective, pretty superficial thing. And
[27:26] Speaker 3: then it had a little footnote on the bottom being like,
[27:28] Speaker 3: as far as I can tell, this doesn't appear in
[27:29] Speaker 3: the literature. Like, would you like me to convert this
[27:31] Speaker 3: into the template for a journal? It was an interesting,
[27:34] Speaker 3: I think it explains a lot what's going on in
[27:36] Speaker 3: my email.
[27:36] Speaker 2: Wait, so you can imagine a future where I guess
[27:39] Speaker 2: they're like, you're sort of overwhelmed with new proofs and
[27:44] Speaker 2: the bottleneck becomes the ability to actually verify them with humans.
[27:48] Speaker 3: That is a problem right now. And in fact, it's
[27:50] Speaker 3: not just in mathematics, it is across academia. So to
[27:53] Speaker 3: give you an example, actually, interestingly, I would say the
[27:56] Speaker 3: academic field that's been affected most by this phenomenon you're
[27:59] Speaker 3: talking about is actually AI and machine learning itself. So
[28:03] Speaker 3: in that particular corner of academia, research is driven by conferences.
[28:06] Speaker 3: So you write your research paper, and there's a sort
[28:09] Speaker 3: of annual cycle where you submit and it undergoes peer review,
[28:12] Speaker 3: and the whole community spends like a month or two
[28:14] Speaker 3: reviewing each other's papers, sort of like, you know, you
[28:17] Speaker 3: pass your paper to the left in class and grade.
[28:19] Speaker 3: It's sort of a very complicated version of that. If
[28:22] Speaker 3: you look at the number of submissions to the machine
[28:24] Speaker 3: learning conferences, it is wild. So in the last 10 years,
[28:27] Speaker 3: I think about 10 years ago, it was maybe one
[28:29] Speaker 3: or 2,000, the deadline for ICLR, the International Conference on
[28:34] Speaker 3: Learning Representations, one of the big ones, I think it
[28:37] Speaker 3: had 60,000 submissions. And if you think about it, it
[28:39] Speaker 3: completely breaks the academic system the way that we're used
[28:42] Speaker 3: to thinking about it. Yeah. We don't have bandwidth to,
[28:46] Speaker 3: never mind just check all of these proofs, even open
[28:48] Speaker 3: up the PDF. Yeah. And so you're then relying on
[28:51] Speaker 3: extremely superficial review process. There's many anecdotes of like high
[28:56] Speaker 3: school students that have somehow gotten involved in peer review
[28:58] Speaker 3: because they had a project once and the system decided
[29:01] Speaker 3: they were in okay. You know, which is frustrating to
[29:04] Speaker 3: build an academic career. Yeah. Essentially, some of the stamps
[29:08] Speaker 3: of quality that we're used to in the field, you
[29:10] Speaker 3: really can't rely on for exactly the reason you identified.
[29:13] Speaker 3: The way that Terence Tao put it is that it
[29:14] Speaker 3: used to be we were in this era of proof scarcity,
[29:16] Speaker 3: that mathematicians, it was really hard. And it was this
[29:20] Speaker 3: very bespoke object that took a lot of craftwork and
[29:23] Speaker 3: training to produce. Now we're in this era of proof abundance.
[29:26] Speaker 3: So now we have to kind of think about different
[29:28] Speaker 3: ways to do our job and to do mathematics in
[29:31] Speaker 3: a productive and interesting way.
[29:33] Speaker 1: Yeah, I think a lot of fields are dealing with
[29:38] Speaker 1: very similar issues. in which we used to have like certain,
[29:42] Speaker 1: I mean, even take writing, for example. We used to
[29:45] Speaker 1: have these certain heuristics that we, maybe like shortcuts that
[29:50] Speaker 1: we use. It's like, okay, if this person can like
[29:54] Speaker 1: construct a series of, like, well-written sentences.
[29:58] Speaker 3: We might.
[29:59] Speaker 2: And no spelling errors.
[30:00] Speaker 1: Yeah. Like, how I've... I never make spelling errors. We
[30:04] Speaker 1: might start... You know, we start from this... Okay, this
[30:06] Speaker 1: person might be worth, like, reading. They might have something...
[30:09] Speaker 1: You know, we all have to, like, operate with these
[30:11] Speaker 1: cheats and shortcuts.
[30:12] Speaker 3: That's right.
[30:13] Speaker 1: And now, like, obviously, producing text with no spelling errors
[30:16] Speaker 1: or anything is available to literally everyone. So it's like
[30:19] Speaker 1: we have to go a step further, et cetera.
[30:21] Speaker 3: I think that's right.
[30:22] Speaker 1: And so, like, You know, all these like heuristics, you know,
[30:24] Speaker 1: speaking of like everyone now, like thinking they can come
[30:28] Speaker 1: up with a proof. Do you remember last year, Travis Kalanick,
[30:32] Speaker 1: he was the Uber founder. He was on the All
[30:36] Speaker 1: In podcast and he was talking about how he was
[30:39] Speaker 1: he was doing vibe physics with Brock.
[30:41] Speaker 3: Oh, boy.
[30:41] Speaker 1: And he felt that he was, quote. Getting pretty damn
[30:45] Speaker 1: close to some interesting breakthroughs. Vibe physics was.
[30:49] Speaker 2: The crazy thing is he could just hire a bunch
[30:51] Speaker 2: of physicists to like verify the research and then you'd
[30:54] Speaker 2: have a real sycophancy problem.
[30:56] Speaker 3: And I think beyond that, like we're currently, so I
[30:58] Speaker 3: work in education. Yeah. One of the things that we
[31:01] Speaker 3: really rely on is somehow the friction in the creative
[31:05] Speaker 3: process and the technical process. I mean, you mentioned writing
[31:07] Speaker 3: is a fantastic example. How do you learn how to write? Well,
[31:10] Speaker 3: you do a lot of writing. And now it's very
[31:13] Speaker 3: difficult to convince our students to sit down and write
[31:16] Speaker 3: an essay because they have these AI tools that can
[31:18] Speaker 3: do it. But then writing a five-paragraph essay was never
[31:22] Speaker 3: the goal. It was to teach you to do other stuff.
[31:24] Speaker 3: Mathematics is similarly experiencing a very similar challenge.
[31:28] Speaker 1: So let's talk about the field of education itself.
[31:32] Speaker 3: I mean.
[31:34] Speaker 1: Are the students very anxious right now?
[31:36] Speaker 3: Absolutely.
[31:36] Speaker 1: Let's start there. What's the vibe among students, especially maybe
[31:41] Speaker 1: grad students who started their career before the idea of
[31:45] Speaker 1: AI-generated resources? The thing is suddenly they're in grad school.
[31:49] Speaker 1: What's going through their minds these days?
[31:51] Speaker 3: Yeah, the landscape of both graduate and undergraduate education is
[31:55] Speaker 3: changing a lot. Now, in the graduate school world, for me,
[31:58] Speaker 3: that would mean PhD. In other worlds, that's maybe master's
[32:01] Speaker 3: degrees and so on. They're It sort of divides into
[32:05] Speaker 3: maybe professional education, preparing you for a job, and then
[32:09] Speaker 3: maybe academic style, like PhD work. In the PhD world,
[32:12] Speaker 3: of course, they're being strongly affected by just the larger
[32:14] Speaker 3: trends and changes in the research universe. And then when
[32:18] Speaker 3: it comes to professional degrees, the main question we have
[32:20] Speaker 3: to ask is, what is the right thing to teach
[32:22] Speaker 3: our students in the current moment? So, for example, I'm
[32:24] Speaker 3: in a department that teaches computer science. The field of
[32:27] Speaker 3: computer science is changing by the day, right? It used
[32:29] Speaker 3: to be, for example, we talked about programming as this
[32:31] Speaker 3: core skill and thing that you should... Everybody should learn
[32:35] Speaker 3: from day one. I would argue that's probably still the case,
[32:38] Speaker 3: the same way that maybe learning to write and be
[32:41] Speaker 3: literate is a key part of being a person in
[32:43] Speaker 3: the society, even if you're not doing that every day.
[32:46] Speaker 3: But jobs as an actual programmer are becoming more rare, right?
[32:49] Speaker 3: We see a decline, for example, in enrollment in computer
[32:53] Speaker 3: science for the first time in like 20 years.
[32:55] Speaker 1: Since you mentioned computer science and unsolved questions, what is
[33:00] Speaker 1: P equals NP all about?
[33:03] Speaker 3: I see, I can answer all your math questions today, absolutely.
[33:05] Speaker 1: These are just all these questions I've stored up in
[33:08] Speaker 1: my head.
[33:08] Speaker 3: Right, and we should come back to, I don't think
[33:10] Speaker 3: I actually told you the end of the Navier-Stokes story,
[33:12] Speaker 3: so I can come back to that. But yeah, so
[33:14] Speaker 3: P versus NP and Navier-Stokes are examples of very famous
[33:17] Speaker 3: mathematical problems that have been identified by the Clay Math
[33:20] Speaker 3: Institute as sort of this benchmark for progress in modern mathematics, right?
[33:24] Speaker 3: So they're open questions we don't know how to answer.
[33:26] Speaker 3: P versus NP is a computer science problem in the
[33:30] Speaker 3: sense that it's basically asking are all problems solvable in
[33:34] Speaker 3: polynomial time? Meaning can your computer solve, or all is
[33:37] Speaker 3: a little strong, but there's certain classes of problems that
[33:40] Speaker 3: we know are solvable pretty efficiently. So for example, your
[33:43] Speaker 3: computer can sort a list of numbers really fast. There
[33:46] Speaker 3: are other problems that we don't know how to solve efficiently,
[33:48] Speaker 3: but that's different from saying we can't solve them efficiently.
[33:52] Speaker 3: And so that's the sort of fundamental question in that universe,
[33:54] Speaker 3: like do there exist problems Is this.
[33:57] Speaker 2: Like the known unknowns and unknown unknowns equivalent of the
[34:01] Speaker 2: computer world?
[34:02] Speaker 3: Yeah, yeah. It's hard to communicate about that way. That's
[34:04] Speaker 3: absolutely right. In fact, I loved... There was... Years ago,
[34:07] Speaker 3: there was a really terrible movie that came out that
[34:10] Speaker 3: involved like P versus NP and that secretly some mathematician
[34:13] Speaker 3: had proved that they were equal, but there's a government
[34:15] Speaker 3: conspiracy because the markets were all gonna collapse. I think
[34:19] Speaker 3: it was called Traveling Salesman. Anyway, the reviews were really funny.
[34:24] Speaker 3: They were like, it turns out it's really hard to
[34:25] Speaker 3: make drama out of people in a room at a blackboard.
[34:28] Speaker 3: That aside.
[34:29] Speaker 2: They should have used Windows instead.
[34:30] Speaker 3: That's right. Yeah, yeah. It would have made it a
[34:32] Speaker 3: little visually compelling. Right. So a lot of these questions
[34:34] Speaker 3: are sort of yes or no. We don't know if
[34:36] Speaker 3: a fact is true. Navier-Stokes is another one. Either you
[34:39] Speaker 3: need to show that these equations have a solution for
[34:41] Speaker 3: all time, like fluid moving around, or give a counterexample
[34:45] Speaker 3: where they don't. So in this AI case, for example,
[34:47] Speaker 3: what the AI gave was a counterexample. It said this
[34:49] Speaker 3: conjecture is not true, that there exists a particular configuration
[34:54] Speaker 3: of your fluid that becomes infinitely turbulent in a finite
[34:57] Speaker 3: amount of time.
[34:57] Speaker 2: With a special spoon.
[34:59] Speaker 3: That's right, with a very special spoon. And the reason
[35:01] Speaker 3: I'm highlighting that is because I think it's a really
[35:03] Speaker 3: great example of the really complicated relationship between humans and AI.
[35:08] Speaker 3: So the construction of that spoon arguably was done in
[35:13] Speaker 3: large part by a team in Spain of human mathematicians
[35:16] Speaker 3: on pen and paper. And then essentially what happened was
[35:19] Speaker 3: the AI, there was also this sort of interpersonal fight
[35:21] Speaker 3: between two different AI teams. That's a whole hot mess.
[35:24] Speaker 3: But that aside... these two teams were sort of, in
[35:27] Speaker 3: some sense, leveraging these guys' progress over a very long
[35:30] Speaker 3: period of time and then doing that last mile, right?
[35:33] Speaker 3: So it's true that the AI made some really critical
[35:36] Speaker 3: insights in that particular counter example, but it isn't true
[35:39] Speaker 3: that it's like a totally surprising result that came out
[35:41] Speaker 3: of left field.
[35:42] Speaker 2: Well, so this seems like it's going to be a
[35:43] Speaker 2: scenario that academia is going to have to deal with
[35:46] Speaker 2: more and more. So you could have a theoretical situation
[35:49] Speaker 2: where like one person comes up with the problem, another
[35:53] Speaker 2: or a team, one person or a team comes up
[35:55] Speaker 2: with the solution. And then you have a model that
[35:57] Speaker 2: like generates the verification or the final leg of the
[36:01] Speaker 2: problem or whatever. Who deserves credit in that scenario? Like,
[36:06] Speaker 2: how do you actually handle that?
[36:08] Speaker 3: How do we handle that currently? Poorly. And it's a
[36:10] Speaker 3: really interesting and challenging discussion. I don't think we have
[36:12] Speaker 3: a good answer right now. It used to be, for example,
[36:16] Speaker 3: these clay math prizes, they're at the very top of
[36:18] Speaker 3: the universe there. I'm not going to get one. I'm
[36:20] Speaker 3: not going to get anywhere close. It used to be
[36:22] Speaker 3: that they were sort of signifiers of two things. One
[36:24] Speaker 3: is that the person that proved the theoretical result is
[36:28] Speaker 3: a really smart person. The other one is that there's
[36:31] Speaker 3: a big achievement in mathematics itself that we now know
[36:34] Speaker 3: something to be true that we didn't know before or false.
[36:37] Speaker 3: Now it's not clear that those two things are correlated,
[36:40] Speaker 3: as you say, right? Like if a computer generates the
[36:42] Speaker 3: interesting part of a proof, who deserves credit? Should that
[36:46] Speaker 3: person be hired if they're the one that made the prompt?
[36:48] Speaker 3: It really is an interesting question for a math or
[36:50] Speaker 3: an engineering department. And it's not so clear. I think
[36:53] Speaker 3: it really depends on your value systems. Like, what do
[36:55] Speaker 3: we get out as humans from this exercise of proofwriting?
[36:59] Speaker 3: I mean, for example, does a counterexample to Navier-Stokes, is
[37:02] Speaker 3: that going to revolutionize the markets or change the way
[37:05] Speaker 3: we write software? Almost certainly not, right? So this is
[37:08] Speaker 3: more of a demonstration of a technology than any particular
[37:11] Speaker 3: practical thing. And so I think we have to be
[37:13] Speaker 3: really careful with understanding the incentive system behind who's claiming
[37:17] Speaker 3: for credit and also making sure that the humans that
[37:20] Speaker 3: are maybe doing all of the labor of composing the
[37:23] Speaker 3: data that's going into these systems that's getting remixed into
[37:26] Speaker 3: a proof, somehow that we're able to make some attribution there,
[37:29] Speaker 3: which currently the technology is really bad at, right? Like
[37:31] Speaker 3: you talk to AI, it's not so clear where it's
[37:34] Speaker 3: getting its answers from, even though we know that it's
[37:36] Speaker 3: kind of echoing data that's maybe scraped off the internet
[37:40] Speaker 3: or someone else's research paper or what have you.
[37:42] Speaker 2: Right, and this might be like the... the time that
[37:46] Speaker 2: we will know that for sure. And five years from now,
[37:49] Speaker 2: like who knows where it's getting its data from? It
[37:51] Speaker 2: could be scraping it from itself or like other models
[37:54] Speaker 2: are coming up with its own stuff. I have a
[37:56] Speaker 2: slight aside, which is the prizes that you mentioned, which
[38:01] Speaker 2: have also come up in movies like Good Will Hunting.
[38:05] Speaker 2: The prize money that people get for that, is that-
[38:09] Speaker 2: For them personally, or is it supposed to go into
[38:12] Speaker 2: their research?
[38:13] Speaker 3: That's a good question. I have no idea. The Claiming
[38:17] Speaker 3: Award specifically, I'm not sure.
[38:20] Speaker 1: Because some of them are big.
[38:21] Speaker 2: And so now I'm envisioning the scenario where like maybe
[38:24] Speaker 2: one day we have a model that ends up winning
[38:27] Speaker 2: some big mathematics prize and the money... goes to the lab.
[38:32] Speaker 3: I guess? I don't know.
[38:33] Speaker 1: I need the $ 1 million.
[38:34] Speaker 2: Well, no, I'm just wondering.
[38:37] Speaker 3: It's actually a good question, and we now have an
[38:39] Speaker 3: example of that. So this Navier-Stokes problem, I should be clear,
[38:43] Speaker 3: so their proof has not been officially verified and checked,
[38:48] Speaker 3: and in particular, there's a rule that it has to
[38:51] Speaker 3: kind of exist in the community for some number of
[38:53] Speaker 3: years and not be refuted. But we have to ask
[38:56] Speaker 3: that question right now. So OpenAI... arguably because they rushed
[39:01] Speaker 3: at the last second, were maybe the first to release
[39:04] Speaker 3: something that satisfies the Clay criteria. There's a kind of
[39:08] Speaker 3: a choose-your-own-adventure situation. There's a few different conjectures, and you
[39:11] Speaker 3: could prove any one of them and then get this
[39:13] Speaker 3: particular prize. OpenAI was quick to say that they don't
[39:16] Speaker 3: need the million dollars, because of course they don't. In
[39:18] Speaker 3: two minutes, they're burning through a million dollars worth of
[39:20] Speaker 3: compute anyway. Which I think begs the question, like, well,
[39:23] Speaker 3: why are they doing this? It's not for the money. Yeah.
[39:26] Speaker 3: And now I don't think it was true that it
[39:27] Speaker 3: was for the money for the mathematicians either. But the
[39:29] Speaker 3: million dollars made a big difference to an individual.
[39:32] Speaker 1: There was some story about some Russian mathematician who sold
[39:35] Speaker 1: one of those big prizes, right?
[39:36] Speaker 3: And then declined the award altogether. He also declined the money.
[39:39] Speaker 3: That's right.
[39:39] Speaker 2: I'm not going to decline the money, for the record.
[39:42] Speaker 3: No, I would happily take it.
[39:44] Speaker 2: When I finally solve some unprovable mathematical theorem. But actually,
[39:48] Speaker 2: this leads to another more serious question, which is, okay,
[39:52] Speaker 2: it costs a lot of money for these models to
[39:55] Speaker 2: be run, and they consume vast amounts of tokens from
[39:57] Speaker 2: very advanced mathematics. Do you end up in a situation
[40:02] Speaker 2: where whoever has the most tokens wins the mathematics prize?
[40:06] Speaker 3: Arguably. That, I think, is one of the major concerns
[40:08] Speaker 3: in our community right now. And by the way, you
[40:11] Speaker 3: mentioned education. It's also a concern there. Does a student
[40:13] Speaker 3: that has a $ 200 account get to have a better
[40:15] Speaker 3: homework solution? Yeah. It's a real question we have to
[40:17] Speaker 3: grapple with right now. On the advanced end, I think
[40:20] Speaker 3: we saw that. So, for example, I don't think it's
[40:23] Speaker 3: controversial to say that essentially when the news that people
[40:26] Speaker 3: were getting close to this Navier-Stokes counterexample came out, it
[40:30] Speaker 3: appears that OpenAI suddenly poured some ridiculous amount of tokens
[40:34] Speaker 3: or money or whatever into writing their proof first. The
[40:38] Speaker 3: typical academic can't do that.
[40:39] Speaker 1: And I think there's an important element, I believe, with
[40:43] Speaker 1: an unreleased model. And so I'm curious, let's say you're
[40:48] Speaker 1: a mathematician and you're an enthusiast. Let's say you're, you
[40:51] Speaker 1: know what, I'm going to embrace this new world of
[40:54] Speaker 1: AI augmenting our understanding of mathematics. If you're a mathematician
[41:00] Speaker 1: at a university, it's certain that there is a model
[41:04] Speaker 1: that exists inside the labs that's better than the model
[41:06] Speaker 1: that you have access to.
[41:08] Speaker 3: That's right.
[41:08] Speaker 1: And so it's like you start to work on some problems.
[41:11] Speaker 1: You start to demonstrate that actually using AI, you can
[41:15] Speaker 1: make progress towards this. And then suddenly it's like, well,
[41:19] Speaker 1: they have a model two generations ahead of you. And
[41:21] Speaker 1: then they knock it out in a week.
[41:23] Speaker 3: And by the way, for all you know, their model
[41:24] Speaker 3: was trained partially on your chats, right? It's really pernicious
[41:28] Speaker 3: if you think about it. That's right. I mean, at
[41:31] Speaker 3: least in the current moment, the AI system is very centralized, right?
[41:34] Speaker 3: I mean, there's two or three tools that almost everybody
[41:37] Speaker 3: is using. And they're owned by these companies. When you
[41:42] Speaker 3: write your query, it goes to their servers. And they're
[41:44] Speaker 3: basically the guys controlling the faucet. is a big concern.
[41:47] Speaker 3: Another issue related to that is research funding. As your
[41:51] Speaker 3: listeners may or may not know, even independently of AI,
[41:54] Speaker 3: research funding currently is in a bit of a crisis.
[41:57] Speaker 3: I mean, it's in a much more challenging moment than
[41:59] Speaker 3: it was in the past. I can tell you, I'm
[42:01] Speaker 3: so fortunate to work at MIT. And even for me,
[42:04] Speaker 3: it's a challenge to pay for my graduate students and
[42:06] Speaker 3: so on. And the costs change every year. Now we
[42:08] Speaker 3: have to add onto that, for example, AI accounts. And
[42:12] Speaker 3: that sounds theoretical, but it's not. My PhD students, for example,
[42:15] Speaker 3: want the most expensive possible cloud account, which is great,
[42:18] Speaker 3: but it's, what, $ 200 per person per month almost becomes
[42:24] Speaker 3: equitable with the cost of another human. Now, something with.
[42:27] Speaker 1: I read, apparently, some people had said that of the
[42:31] Speaker 1: outstanding clay prizes, many had sort of surmised that Navier-Stokes
[42:37] Speaker 1: might be the first one solved. And I think it's interesting...
[42:41] Speaker 1: that it was solved via counter example. And it sort
[42:46] Speaker 1: of raises a question of like, did the model find
[42:50] Speaker 1: an elegant solution or did it just grind? Was this
[42:53] Speaker 1: a grindable problem?
[42:55] Speaker 3: That's exactly the right question.
[42:56] Speaker 1: Or just a grind through a lot of states and
[42:58] Speaker 1: then it hits on the magic spoon that disproves it
[43:02] Speaker 1: or et cetera. Which seems like a very, like if
[43:04] Speaker 1: finding that counter example is a process of grinding through
[43:10] Speaker 1: hundreds of thousands of different states. Like, is that elegant
[43:14] Speaker 1: math or is that just like some sort of like
[43:16] Speaker 1: sheer mechanical process?
[43:18] Speaker 3: Oh, that's a great question. And indeed, I think you
[43:20] Speaker 3: mentioned the beginning of this, this talk that you opened
[43:22] Speaker 3: that paper and it's like totally unreadable. And that's not
[43:25] Speaker 3: just whether you have a math degree or not. Like
[43:27] Speaker 3: I can't read past the first couple of pages, you know? Right.
[43:30] Speaker 3: So that particular result, I think there was some beautiful
[43:33] Speaker 3: insight again, arguably generated by humans, that went into the
[43:37] Speaker 3: overall construction. And then indeed, the proof is really horrifying.
[43:40] Speaker 3: Like there's a lot going on. It's technical. It's got
[43:42] Speaker 3: inequalities and bounds and indices and all kinds of crazy stuff.
[43:46] Speaker 3: And so I think indeed, as you say, in this
[43:48] Speaker 3: particular case, no, is it elegant? Probably not. Is it correct? Yes, probably.
[43:54] Speaker 3: So again, basic question, like why do we do this exercise?
[43:57] Speaker 3: Is it just to now we can say with confidence
[44:00] Speaker 3: that this conjecture is false? Or is it that we
[44:03] Speaker 3: learn something along the way that'll bring insight to some
[44:05] Speaker 3: other problem? Those two things are now divorced, right? If
[44:08] Speaker 3: you just have this frictionless way to prove a theorem,
[44:11] Speaker 3: then all the beautiful insight and maybe kind of fun
[44:14] Speaker 3: garden paths along the way that might have inspired some
[44:17] Speaker 3: new research field have been pruned out, which is a
[44:20] Speaker 3: big challenge, right? In some sense, that's really the key
[44:23] Speaker 3: human aspect of all of this.
[44:40] Speaker 2: So this was going to be my next question. And
[44:44] Speaker 2: I guess I'm really asking you to opine on whether
[44:46] Speaker 2: or not you think AGI is going to be a
[44:48] Speaker 2: thing or super intelligence or whatever we're calling it now.
[44:51] Speaker 2: But OK, so AI right now can sort of brute
[44:55] Speaker 2: force a solution. And it's not particularly elegant. Can it
[45:01] Speaker 2: come up with ideas that are separate to what we
[45:05] Speaker 2: kind of already know or questions that we are already positing? Like,
[45:09] Speaker 2: can it actually inform us that, no, the way you
[45:12] Speaker 2: think the world works currently is actually different? Or can
[45:16] Speaker 2: it sort of like unveil mathematical rules or rules about
[45:19] Speaker 2: the world that we are not even thinking about or
[45:23] Speaker 2: asking about?
[45:23] Speaker 3: New conjecture.
[45:24] Speaker 2: Yeah, exactly.
[45:25] Speaker 3: Exactly. It's a great question. So actually, my colleague Nestor
[45:28] Speaker 3: Guillen at NYU has a nice blog post that kind
[45:31] Speaker 3: of talks about this a bit. When you look at
[45:33] Speaker 3: a lot of the proofs and the new results that
[45:37] Speaker 3: are coming out of these AI systems, they're arguably in
[45:39] Speaker 3: what mathematicians might call the convex hole of existing knowledge,
[45:42] Speaker 3: meaning there's sort of a bunch of stuff we already know.
[45:45] Speaker 3: And then a lot of these proofs are really intricate
[45:47] Speaker 3: and interesting ways to mix and match those things. They're
[45:49] Speaker 3: like kind of inside of the span of knowledge that
[45:52] Speaker 3: we already have. And you've asked the key question, which
[45:54] Speaker 3: is can a mathematical AI system actually come up with
[45:59] Speaker 3: a truly surprising new result that's like not just mixing
[46:03] Speaker 3: and matching facts we already know, or maybe like finishing
[46:06] Speaker 3: off a calculation that somebody got stuck on, but making
[46:09] Speaker 3: an entirely new theory, for example, that just wasn't present before.
[46:13] Speaker 3: I don't think we've seen a whole lot of examples
[46:15] Speaker 3: of that so far. And now, I mean, this technology
[46:17] Speaker 3: changes by the day, like who knows, but at least
[46:19] Speaker 3: so far, I think for the most part, it's in
[46:21] Speaker 3: that first category of like, maybe making really interesting and
[46:25] Speaker 3: surprising connections between surprising, between, things that we already knew,
[46:29] Speaker 3: but not necessarily generating something entirely new out of left field, right?
[46:33] Speaker 3: That would be really the next step for a lot
[46:34] Speaker 3: of these tools.
[46:35] Speaker 1: I know that like one of the questions too, with
[46:37] Speaker 1: some of the, and I think maybe some of the
[46:39] Speaker 1: lesser prestigious proofs that have been established, there's the question, okay,
[46:43] Speaker 1: maybe they just grind their way to an answer. And
[46:46] Speaker 1: the other, I know another question is like, well, was
[46:49] Speaker 1: this already sort of proven somewhere? Maybe it was like,
[46:51] Speaker 1: there is some obscure Russian math journal where most of
[46:54] Speaker 1: the work had been done and most of them. mathematicians
[46:57] Speaker 1: were aware of that. But it was in the training data.
[47:00] Speaker 1: And so.
[47:01] Speaker 3: Hard to know. Data attribution in AI generally, mathematics or otherwise,
[47:07] Speaker 3: is a really critical problem. I mean, it's exactly the
[47:09] Speaker 3: same problem as like, my AI generated a cool new cartoon.
[47:13] Speaker 3: Did it borrow from Disney? And is that okay? In
[47:16] Speaker 3: some sense, it's a very similar problem. And the current
[47:18] Speaker 3: tools are very bad at answering it.
[47:19] Speaker 1: Why don't you just tell us a little bit about
[47:22] Speaker 1: being a professor in 2026? And like, what you're doing
[47:28] Speaker 1: differently and just that education.
[47:32] Speaker 3: Yes. So the education field is having its own, I
[47:35] Speaker 3: don't know about identity crisis, but certainly a lot of
[47:37] Speaker 3: navel gazing and thinking very hard about the fundamentals. MIT
[47:41] Speaker 3: actually released a really nice little report on AI and education.
[47:45] Speaker 3: I encourage everybody to take a look. My colleagues, Sam
[47:47] Speaker 3: Klopfer and Eric Madden, were in charge of writing that.
[47:50] Speaker 3: But every aspect of education has changed. I think One
[47:54] Speaker 3: thing that maybe is undervalued is the amount of education,
[47:57] Speaker 3: which is social, even if primarily the maybe artifacts that
[48:01] Speaker 3: you're producing are things like problem sets. So one thing
[48:04] Speaker 3: that we've seen among our sort of youngest students, for example,
[48:07] Speaker 3: the early undergraduates, is that it's really hard to draw
[48:10] Speaker 3: them out of their dorm room. There's an interesting perception
[48:12] Speaker 3: challenge here. You now have, again, this frictionless robot that
[48:17] Speaker 3: can help you with your problem sets, or frankly, do
[48:19] Speaker 3: your problem set for you. Is that learning? Probably not. Now,
[48:23] Speaker 3: the interesting thing is that there's a difference between perception
[48:26] Speaker 3: and reality. A lot of our students maybe perceive that
[48:29] Speaker 3: they're learning when they use these different AI tools. Yeah.
[48:32] Speaker 3: But then they go and take the exam and the
[48:34] Speaker 3: score would say otherwise.
[48:35] Speaker 1: So what do you do to sort of maximize the
[48:39] Speaker 1: chances that your students are learning?
[48:41] Speaker 3: You know, at MIT, they have a slogan. I think
[48:43] Speaker 3: it roughly is like mind and hand learning. And then
[48:45] Speaker 3: they added and heart on the end, which is good.
[48:48] Speaker 3: And I think we're really leaning into that second one.
[48:50] Speaker 3: We've known long before AI that putting a bunch of
[48:52] Speaker 3: students in a classroom and talking at them and then
[48:54] Speaker 3: having them work on a problem set alone is not
[48:57] Speaker 3: a very effective way to teach.
[48:58] Speaker 1: Okay.
[48:58] Speaker 3: So now I think we're really focusing on experiential learning,
[49:02] Speaker 3: on getting the students to show up physically and to
[49:04] Speaker 3: ask that question or answer that question rather of why
[49:07] Speaker 3: do we take all these students and put them on
[49:09] Speaker 3: a college campus instead of having them work from a
[49:11] Speaker 3: laptop at home? Now, I would argue that that's still
[49:14] Speaker 3: a really valuable exercise. MIT, for example, has the luxury
[49:17] Speaker 3: of having the world expert on just about anything. physically
[49:20] Speaker 3: within a mile of your dorms. And so we're designing
[49:24] Speaker 3: all kinds of programs that, for example, in the School
[49:26] Speaker 3: of Engineering, try to bring the students in for research experience,
[49:30] Speaker 3: for hands-on learning, for going into industry and seeing what
[49:33] Speaker 3: things look like there. And trying to build the undergraduate
[49:36] Speaker 3: experience more out of that and a little bit less
[49:39] Speaker 3: out of exams and problem sets. Now, there remain a
[49:41] Speaker 3: lot of questions. For example, assessment is a big challenge
[49:44] Speaker 3: for us. What we observed in the last year or
[49:46] Speaker 3: two is that our students are getting nearly 100% on
[49:48] Speaker 3: all their homeworks. Maybe it's because MIT students are super bright.
[49:52] Speaker 2: Can I just say, so I take the train from
[49:56] Speaker 2: New York to between New Haven quite a lot. There
[49:59] Speaker 2: are a lot of Yale students. Watching them on their
[50:02] Speaker 2: laptops right now, they're all on, mostly Claude actually, writing
[50:07] Speaker 2: essays or at least doing the basic research via chatbot.
[50:11] Speaker 3: So like this semester, I'm teaching a course called shape
[50:13] Speaker 3: analysis that covers discrete differential geometry, if you're into that
[50:16] Speaker 3: kind of thing.
[50:17] Speaker 1: Of course.
[50:18] Speaker 3: Who isn't, right? This is a pretty mathy class. In
[50:21] Speaker 3: the past, most of what the students did were two things.
[50:24] Speaker 3: It was kind of written assignments and then an open-ended
[50:27] Speaker 3: project where they kind of come up with a new
[50:28] Speaker 3: result and write it down. And both of those had
[50:31] Speaker 3: to change. And so in our course now, the way
[50:34] Speaker 3: we structure it is there's still homeworks, but The analogy
[50:38] Speaker 3: that I'm using, as with many of my colleagues at MIT,
[50:40] Speaker 3: is closer to weightlifting. You don't do it because it's fun, necessarily.
[50:45] Speaker 3: I do sometimes, but not everybody's like that. But you
[50:49] Speaker 3: do it more because you're trying to achieve some other goal.
[50:52] Speaker 3: You're preparing or you're being a stronger person. And then
[50:55] Speaker 3: maybe what you do is you have to test understanding
[50:57] Speaker 3: in some other fashion. So we have a little short
[50:59] Speaker 3: quiz after the homework that basically is just a copy-pasted
[51:02] Speaker 3: homework problem, but makes sure that the students at least
[51:04] Speaker 3: comprehend what they did. We also had to restructure the
[51:07] Speaker 3: project a bit, add like oral presentation component to it
[51:10] Speaker 3: and bring some more interaction to make sure that essentially
[51:13] Speaker 3: we're educating the human, not just testing the capabilities of Claude, right?
[51:18] Speaker 3: Those are very different things. And I would argue the
[51:20] Speaker 3: real goal of an institution like MIT is to prepare
[51:23] Speaker 3: brilliant students for the next set of problems.
[51:25] Speaker 1: We were talking about how like it's difficult to just
[51:28] Speaker 1: look at the output of the Navier-Stokes solution.
[51:31] Speaker 3: Yes.
[51:32] Speaker 1: Evidently even for mathematicians. And I was thinking, okay, so
[51:36] Speaker 1: what's the future then of math? And I had this
[51:39] Speaker 1: analogy in my head, like we knew how to brew
[51:43] Speaker 1: beer before we understood the science of yeast, right? So
[51:46] Speaker 1: alcohol has been around a long time. And then at
[51:49] Speaker 1: some point people understood and they knew the process, but
[51:52] Speaker 1: they didn't understand why the process worked.
[51:53] Speaker 3: That's right.
[51:54] Speaker 1: And then eventually they discovered yeast and why this sequence works.
[51:58] Speaker 1: And so I thought like, well, maybe like, could it
[52:01] Speaker 1: be that the future of science is, yes, the AI
[52:06] Speaker 1: models are going to find all these answers or be
[52:10] Speaker 1: able to do all these things, but that there will
[52:12] Speaker 1: still be a role then for humans to like, I
[52:15] Speaker 1: don't know, translate it or explain it.
[52:18] Speaker 3: I think that's right.
[52:20] Speaker 1: But then I also wonder if it's like, well, probably
[52:24] Speaker 1: the AI is also going to be better at translating
[52:26] Speaker 1: and explaining things too. But could there be a new
[52:29] Speaker 1: sort of general reorientation of math such that rather than
[52:35] Speaker 1: discovering new things per se, it's like understanding these things
[52:40] Speaker 1: that have been proven?
[52:41] Speaker 3: That's right. I think the right way to think about
[52:44] Speaker 3: these AI systems is that they're tools. And people's jobs
[52:47] Speaker 3: evolve all the time, and apparently mine is no different.
[52:51] Speaker 3: I want to be clear, I don't think that we
[52:52] Speaker 3: need to be subservient to the AI and just checking
[52:53] Speaker 3: what it makes. I guess there's a few different aspects
[52:56] Speaker 3: of that. One is, of course, my job is both
[52:59] Speaker 3: education and research. And I think the education part, even
[53:02] Speaker 3: if what it looks like day to day has changed
[53:03] Speaker 3: a bit, the mission is still there. And we still
[53:05] Speaker 3: want to educate the world's best humans at MIT. And
[53:08] Speaker 3: we do an okay job of that. In terms of
[53:10] Speaker 3: the mathematical labor, I think one thing that's really undervalued
[53:14] Speaker 3: in this field is that the typical mathematician is not
[53:17] Speaker 3: working on the Clay Math Institute problems.
[53:19] Speaker 1: It was really hard.
[53:20] Speaker 3: There's some really brilliant guys. I wish I could do it.
[53:22] Speaker 3: I'm not, I'm not, I can't compete there. It's similarly
[53:24] Speaker 3: like math contest problems was another one that the AI
[53:28] Speaker 3: companies were quick to kind of pounce on.
[53:29] Speaker 1: As the math Olympia.
[53:31] Speaker 3: Exactly. That was the sort of previous generation.
[53:33] Speaker 1: Yeah.
[53:34] Speaker 3: All of these are great in the sense that they're
[53:36] Speaker 3: like problems that you can write down. They're well contained.
[53:38] Speaker 3: They're in a little box, but a lot of mathematics
[53:41] Speaker 3: is more about taste than it is about just proofwriting
[53:45] Speaker 3: and verifying some other conjecture that somebody wrote down. So
[53:48] Speaker 3: for example, in my particular area, and again, I want
[53:51] Speaker 3: to be clear, you know, I'm an applied guy. A
[53:53] Speaker 3: lot of pure mathematicians would absolutely look down on what
[53:55] Speaker 3: I'm doing and say, ah, that's not real math. I
[53:57] Speaker 3: would say otherwise, but that's, we'll have that debate another day.
[54:00] Speaker 3: By the time we get to writing a proof, we
[54:02] Speaker 3: probably already know it's true. Like it's more double check situation.
[54:06] Speaker 3: But the really interesting part is on the modeling and
[54:09] Speaker 3: figuring out which questions to ask and what insight they
[54:11] Speaker 3: bring us to the universe. And the AI tools are
[54:15] Speaker 3: not doing that, right? And so you're right. Maybe proofreading
[54:19] Speaker 3: becomes more of an exercise in writing prompts, but writing
[54:22] Speaker 3: the prompts is very much what we've been teaching our
[54:24] Speaker 3: students to do for centuries now. And that skill, I think, persists. Yeah.
[54:28] Speaker 2: I mean, broadly, it feels like the emphasis is shifting
[54:31] Speaker 2: to being able to ask the right questions, come up
[54:34] Speaker 2: with new ideas versus just come up with a basic answer.
[54:38] Speaker 3: That's right. And the typical math student probably doesn't end
[54:41] Speaker 3: up in theoretical math. They maybe end up in finance.
[54:43] Speaker 3: They end up in engineering. And for those folks, that
[54:46] Speaker 3: skill is the really critical one, right? It's exactly what
[54:49] Speaker 3: you need to be a good manager, to be a
[54:50] Speaker 3: good project designer, to lead and innovate in all kinds
[54:53] Speaker 3: of different fields. And that skill is somehow becoming more
[54:56] Speaker 3: important than ever. All that boring proofreading stuff that everybody's
[54:59] Speaker 3: allergic to is actually kind of being pushed to the
[55:02] Speaker 3: side a bit.
[55:03] Speaker 1: So we should still teach kids math.
[55:05] Speaker 3: I think so.
[55:07] Speaker 1: Justin Solomon, thank you so much for coming.
[55:09] Speaker 2: On the blog.
[55:10] Speaker 3: Of course, it was a pleasure.
[55:18] Speaker 1: Well, I think, look, to say the least, the future
[55:29] Speaker 1: of sort of math and math education, it's like, we
[55:32] Speaker 1: have no idea where this is going, right? Because the
[55:34] Speaker 1: models are going to keep getting better. And even just
[55:37] Speaker 1: the sort of like prosaic point about like, okay, here
[55:41] Speaker 1: you have some conference that used to have a thousand
[55:43] Speaker 1: submissions and now it has 60,000 submissions. And some of
[55:47] Speaker 1: them might even be very good, but it's like, that's
[55:51] Speaker 1: the type of thing that just like, as he said,
[55:53] Speaker 1: it like breaks existing systems pretty straightforward.
[55:56] Speaker 2: Yeah. It feels like there's a tautology here with AI where,
[56:00] Speaker 2: Because of AI, a lot of things are getting easier
[56:02] Speaker 2: and more difficult at the same time, right? So you
[56:05] Speaker 2: have the ability to generate tons and tons of content,
[56:08] Speaker 2: but the ability to verify it doesn't necessarily go up
[56:12] Speaker 2: at the same amount of time. Maybe it will one day.
[56:15] Speaker 2: I also think the emphasis on questions rather than simple
[56:19] Speaker 2: answers and the idea of taste, even in something like mathematics,
[56:23] Speaker 2: which is kind of alien to me. You know, you
[56:25] Speaker 2: hear about it in code all the time, the ability
[56:27] Speaker 2: to write elegant code, and I guess the same applies
[56:30] Speaker 2: to mathematics, the ability to elegantly prove a formula or
[56:34] Speaker 2: elegantly come up with a theory. That seems to be
[56:38] Speaker 2: repeated throughout everything nowadays. If you're able to generate ideas...
[56:44] Speaker 2: think outside the box to use a terrible cliche and
[56:48] Speaker 2: curate with some taste or write or perform with a
[56:53] Speaker 2: particular amount of taste. That seems to be the edge
[56:56] Speaker 2: case here.
[56:56] Speaker 1: I would also say like, this is actually a pretty
[56:58] Speaker 1: high stakes question, even outside of the academic setting sort
[57:03] Speaker 1: of research taste, because of course, one of the big
[57:06] Speaker 1: questions for AI period is this idea of, recursive self-improvement,
[57:13] Speaker 1: can a model build a successor model? And to do
[57:18] Speaker 1: that means generating new ideas to then experiment on. And
[57:24] Speaker 1: I think it's a pretty big open question still whether
[57:28] Speaker 1: the models are good at novel questions, right? So it's like, okay,
[57:34] Speaker 1: there's a bunch of outstanding questions. There's reason to think
[57:38] Speaker 1: they're solvable. okay, maybe the models get very good at that.
[57:43] Speaker 1: Can the model come up with the next Navier-Stokes? Can
[57:48] Speaker 1: it come up with the next P equals NP? Can
[57:50] Speaker 1: it come up with the next Fermat's last theorem that's
[57:52] Speaker 1: in math? But whatever the equivalence of that is in
[57:55] Speaker 1: sort of areas like machine learning and so forth, it's
[57:59] Speaker 1: a pretty important question for the future of AI. It's like,
[58:03] Speaker 1: are the models good at proposing the next research experiment
[58:08] Speaker 1: And then, of course, carrying out, conducting the test.
[58:12] Speaker 3: Right.
[58:12] Speaker 2: And they might be one day. They might be.
[58:15] Speaker 1: But right now, there's clearly still a pretty significant role
[58:19] Speaker 1: for a human in the lube.
[58:20] Speaker 3: Right.
[58:20] Speaker 1: To sort of like, I guess that's intuition or whatever. Imagination. Okay,
[58:26] Speaker 1: let's try this out, et cetera. And my impression is
[58:30] Speaker 1: that the models still aren't great at that particular aspect of,
[58:35] Speaker 1: I guess, the scientific method.
[58:36] Speaker 2: Yeah, I think that's what Justin said as well. So
[58:40] Speaker 2: that gets me back to this idea that the human
[58:42] Speaker 2: edge remains the idea generation, out of the box thinking,
[58:47] Speaker 2: whatever you want to call it, out of the AI box.
[58:50] Speaker 1: Out of the chip thinking.
[58:51] Speaker 2: That's out of the chip thinking.
[58:53] Speaker 3: That's good. Yeah.
[58:53] Speaker 2: Shall we leave it there?
[58:54] Speaker 1: Let's leave it there.
[58:55] Speaker 3: All right.
[58:55] Speaker 2: This has been another episode of the Odd Thoughts Podcast.
[58:58] Speaker 2: I'm Traci Allaway. You can follow me at Traci Allaway.
[59:00] Speaker 1: And I'm Joe Weisenthal. You can follow me at The Stalwart.
[59:04] Speaker 1: Follow our producers, Carmen Rodriguez at Carmen Armit, Dashiell Bennett
[59:07] Speaker 1: at Dashbot, Kale Brooks at Kale Brooks, and Kevin Lozano
[59:11] Speaker 1: at Kevin Lloyd Lozano. And you can check out all
[59:13] Speaker 1: of these topics 24-7 in our Discord, discord.gg slash oddlots.
[59:18] Speaker 1: And for more oddlots content, check out our daily newsletter.
[59:21] Speaker 1: You can find that at bloomberg.com slash oddlots.
[59:24] Speaker 2: And if you enjoyed this conversation, if you like it
[59:26] Speaker 2: when we talk about mathematics, then please let us know
[59:29] Speaker 2: on your favorite podcast platform. If you're watching on video
[59:32] Speaker 2: and you enjoyed the show, then please leave us a
[59:34] Speaker 2: comment or like the video or better yet, subscribe.
[59:37] Speaker 1: And remember, if you're a Bloomberg subscriber, you can listen
[59:40] Speaker 1: to all of our episodes absolutely ad-free. All you need
[59:44] Speaker 1: to do is find the Bloomberg channel on Apple Podcasts
[59:47] Speaker 1: and follow the instructions there.
[59:49] Speaker 2: Thanks for joining.
[59:59] Speaker 3: Thank you.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.