# The Double-Edged Sword of AI, with Dr. Ziad Obermeyer

An episode of [NEJM AI Grand Rounds](https://invisiblenetwork.ai/podcast/1657518313), 27 Jul 2023, 1 h 14 min.

Audio: https://mcdn.podbean.com/mf/web/tyng4p/Episode_8_Obermeyer_2.mp3

In this thought-provoking episode, Dr. Ziad Obermeyer delves into the complex issues of bias, safety, and generalizability of medical AI. Dr. Obermeyer emphasizes the importance of machine learning researchers’ task formulation, an often-overlooked yet significant determinant of bias in AI algorithms. Highlighting the dual impact of machine learning, he compares two of his works that demonstrate how AI can either exacerbate or help mitigate health care disparities. Lastly, he discusses the significant challenges encountered in the development of AI models due to siloed and inaccessible data, sharing his own experiences and solutions in tackling this issue. Dr. Obermeyer is the Blue Cross of California Distinguished Professor at the Berkeley School of Public Health, Co-Founder of Nightingale Open Science, and Co-Founder of Dandelion Health. Transcript

## Transcript

[0:04] I, I think there's this tendency, everyone gets angry at the algorithms.
[0:07] It's like, oh, this algorithm is by, but the algorithm, the algorithm is
[0:10] doing exactly what we told it to do.
[0:12] It's not the algorithm's fault that costs are lower for black
[0:15] patients than for white patients at the exact same level of health.
[0:19] That's our fault.
[0:19] That's society's fault.
[0:21] But now the algorithm is mirroring that deep bias in society because we chose.
[0:27] To have the algorithm mirror that deep bias in society.
[0:30] So that's where the bias came from.
[0:36] That was Dr.
[0:37] Ziad Obermeyer of Berkeley, reflecting on his research that dissected the source
[0:41] of racial bias in a major healthcare algorithm that he published in a landmark
[0:46] paper in the Journal Science back in 2019.
[0:49] I wanna welcome you, our listeners to another episode
[0:52] of N E J M AI Grand Rounds.
[0:54] I'm your co-host Raj Manrai, and I'm delighted to be here
[0:57] with my co-host Andy Beam.
[0:59] Andy, I always really enjoy speaking with Ziad.
[1:02] He's truly creative and rigorous in his research, and I also find him to be an
[1:06] amazing communicator of complex ideas.
[1:08] In addition to enjoying hearing about his pioneering work on algorithmic
[1:12] bias and medical decision making, I thought his comments about the
[1:15] recipe for great interdisciplinary work were really thought provoking.
[1:18] So to do truly creative and impactful work in medical ai, can you pair
[1:22] a great clinician with a great AI researcher, or do the two skills really
[1:25] have to co-exist within the same mind?
[1:28] I think Ziad, both articulates and himself, is a strong argument
[1:31] for the skills having to co-exist.
[1:33] And all in all, this was a really fascinating discussion.
[1:36] I totally agree, Raj, and that was really well said.
[1:39] And I, you know, when I think of Ziad, I think of him as kind of a renaissance man.
[1:43] His work cuts across so many different disciplines from medicine to economics,
[1:48] and as we saw in this episode, he really investigates how these
[1:51] methods impact the lives of patients.
[1:53] And as you mentioned, he has this really unique ability to access
[1:57] and distill very complicated topics in a way that I find inspiring.
[2:01] So it was a great pleasure to talk with him about his research.
[2:05] The N E J M AI Grand Rounds podcast is sponsored by Microsoft and VI ai.
[2:10] We thank them for their support.
[2:13] And with that, we're excited to bring you Dr.
[2:15] Ziad Obermeyer on AI Grand Rounds.
[2:19] Welcome to AI Grand Rounds Ziad.
[2:20] We're super excited to have you on today.
[2:23] Oh, it's, uh, wonderful to be here.
[2:25] Ziad let me also welcome you to the podcast.
[2:27] It's great to have you on.
[2:29] This is a question that we like to get started with, and
[2:31] please forgive our AI puns.
[2:33] Could you tell us about the training procedure for Ziad's neural net?
[2:37] How did you get interested in AI?
[2:39] What data and experiences led you to where you are today?
[2:43] I think the, the proximal experiences are definitely my experience, training
[2:50] and practicing as an emergency physician.
[2:52] I think if you are working anywhere in the medical system and you're
[2:56] vaguely paying attention, you are making mistakes all the time.
[3:00] You're seeing other people making mistakes and you're trying to learn from
[3:04] those errors and minimize your loss.
[3:07] And I think a lot of the things that we need to do as practitioners in
[3:12] the healthcare system are things that look very much like things that
[3:16] machine learning algorithms do well.
[3:18] We're often trying to make diagnoses.
[3:20] What is diagnosis?
[3:21] It's like I look at a patient, I see a bunch of stuff about that patient
[3:25] and I try to convert that into a probability that they have, you know,
[3:29] illness one, illness two, illness three.
[3:32] When I'm working in the emergency department, I'm often trying
[3:36] to think about, well, do I need to bring this patient into the
[3:38] hospital or can I send them home?
[3:40] Are they gonna do okay at home?
[3:42] Um, that's a great prediction problem.
[3:44] That depends on their likelihood of bad things happening to them at home.
[3:47] And so wherever you are looking in medicine, you're seeing these
[3:51] prediction problems like cancer metastasis, great prediction problem,
[3:55] diagnosing the cause of pain on an x-ray, great prediction problem.
[3:59] So I think that's the thing that got me really excited was working as a doctor,
[4:04] seeing all of these really tough decisions that we're having to make doing not
[4:09] as well as I would've liked on those decisions, and trying to find tools that
[4:12] would make me and my colleagues better.
[4:15] If, if I could, hopping on the theme of prediction models, I feel like your
[4:19] career breaks my own prediction model for what physician scientists typically do.
[4:24] Uh, you've written very broadly as we'll talk about on economics, um,
[4:28] things that sort of border on ethics and philosophy, also technical contributions.
[4:33] So could you talk about your training or educational background that
[4:36] sort of helped you tackle such an impressively broad array of subjects?
[4:40] Oh, thank you for the kind words.
[4:42] I, I agree that a large language model would not be able to predict
[4:46] the next word, uh, equivalent in my career because I often really
[4:50] wasn't able to predict it either.
[4:52] So, to go back in time a little bit further before my medical training in
[4:56] college, I actually studied history and philosophy and I did a master's in that.
[5:01] You know, it comes in handy.
[5:03] Sometimes.
[5:03] Um, but it has had this pretty big influence on my worldview.
[5:08] I'm very interested in how new scientific fields form and how that
[5:13] is often a combination of genuine scientific and technical advances,
[5:18] but also a lot of social things and how groups of people come together
[5:22] and build a scientific field.
[5:24] Uh, it's a great example of the human enterprise and all
[5:28] of its beauty and fallibility.
[5:30] So I studied those things.
[5:31] I thought I might wanna be a historian and or a philosopher.
[5:35] It turned out I wasn't that great at either history or philosophy.
[5:38] And so then I thought maybe I'd like to just make a lot of money.
[5:42] And so I went into management consulting and that was actually really fun.
[5:46] It wasn't for me.
[5:47] Uh, I only spent a year doing it.
[5:49] But, but it taught me a lot about how to think strategically and I guess
[5:54] economically, how to think about the trade-offs that people and businesses face
[5:59] and how things are often a lot harder on the inside than they look on the outside.
[6:04] And then I went to medical school, so I think I, I had that, that set of
[6:07] interests and that worldview and the background going into medical school.
[6:12] And I think that might've been one of the things that made me more
[6:14] attuned than other people to the huge opportunity that I see in applying AI
[6:20] and associated tools to, to medicine.
[6:22] Was there something specific about medicine that made your interest
[6:25] in it more durable than some of your other previous interests?
[6:29] It's a great question and part of the reason it, I've thought about this a
[6:33] lot is because I actually really didn't like medical school and so I had I been
[6:38] predicting in year one or year two of medical school, I would not have predicted
[6:41] that I would finish medical school, let alone go on to really love being a doctor.
[6:46] I think there's something about medical school that is a little bit infantilizing
[6:52] you, you just have to memorize a lot of stuff and I think you have to
[6:56] do that for good reason sometimes.
[6:57] But I think there's also this huge mismatch between what you learn in
[7:02] medical school and what you need to be a doctor or a scientist in this area.
[7:08] And so I really struggled with that in medical school and I thought about
[7:12] not doing a residency, but largely I think thanks to inertia and probably
[7:17] not very good decision making.
[7:18] I ended up doing residency anyway.
[7:21] And I had enjoyed the parts of my medical school time that happened in
[7:25] the, um, in the emergency department.
[7:27] So I was like, whatever, I'll just do this.
[7:28] It's a cross section of medicine.
[7:29] So that, you know, I was, I was actually hedging.
[7:32] I was like, well, you know, if I end up not doing medicine later, then
[7:36] at least this will be good training.
[7:37] It'll get me exposed to lots of different parts of medicine.
[7:39] But then it turned out that I really, really liked being a doctor.
[7:44] And I think there's something about that exposure to the
[7:48] real world and the problems of patients that I think it's shaped.
[7:52] The problems that I work on in my research as well.
[7:56] And so I think it's a really important, interesting job.
[7:59] There's drama though it sometimes it does feel like being on a TV show.
[8:03] You go into a room and you can ask anyone anything about their life and
[8:08] they'll answer the question because you're trying to help them, and you
[8:11] have that bond of trust that you forge.
[8:14] It's an amazing job.
[8:15] And then there are all of these really interesting intellectual questions
[8:19] that come off of it that are at the intersection of statistics, economics,
[8:24] behavioral science, computer science.
[8:25] So I just think it's the best job in the world, the job that I'm doing
[8:29] at the intersection of these fields.
[8:31] And I feel incredibly lucky to be doing it.
[8:35] Edd, maybe one more question about your background before we jump into your work.
[8:38] So I believe we actually met through our mutual friend such and Jane, maybe
[8:42] a decade or so ago at a great Cambridge, Massachusetts bar, the Plow and Stars.
[8:47] I was a grad student at the time.
[8:49] I think you were a new faculty member and you were pitching a
[8:51] very interesting project on machine learning for emergency medicine.
[8:55] And so first I have to give you credit as uh, being on the machine learning
[8:59] wave way in advance of much of the field.
[9:02] And I think you've really chosen problems well, uh, and we're gonna discuss some
[9:05] of those projects in detail, but maybe as sort of a background question in
[9:10] general thoughts, I can just ask you to elaborate on some of the cultural
[9:15] differences that you've had to navigate across these different fields.
[9:18] So we've already talked about economics, computer science, medicine, I.
[9:23] These are very different fields in some ways, right?
[9:25] And they have, uh, very different approaches to solving problems.
[9:29] How do you think about that in terms of your training?
[9:32] What have you self learned?
[9:34] What have you formally learned?
[9:35] What have you had to learn on the fly to solve the problems that you do?
[9:40] I think one of the interesting things about medicine as a
[9:45] field is that there's no field.
[9:48] Like there's no, I mean, you go to medical school, but medical school is, it's, it's
[9:53] a little bit like, you know, learning to become a plumber or do some trade.
[9:58] It's like that there's a certain amount of really concrete stuff you have to learn
[10:01] how to do, and you learn how to do it.
[10:03] But then there's this scientific backend to medicine that is very much at the
[10:09] intersection of a lot of these fields.
[10:11] Um, and I think that's, it makes it hard to do this work because
[10:16] unfortunately no one field is gonna give you everything that you need.
[10:21] And so one of the things that I actually admire about both of you and the work
[10:25] that you've done is that you've really invested a lot in learning the content.
[10:29] As the computer scientist say the domain expertise, and I think that's
[10:34] really a great model for doing work in this field, or, or it's
[10:37] at least what I've tried to do.
[10:39] So I, I've spent a lot of time doing like problem sets and textbooks and, you
[10:46] know, we, and this is on your own, right?
[10:48] Out outside of a formal class?
[10:50] Yeah.
[10:50] It, it was, and I think it was because there were just some questions I
[10:53] was really interested in that I wanted to be able to solve myself.
[10:58] I think, you know, it's different.
[11:01] I think what the, the work that both of you do and the work that I try to do.
[11:04] I think it's different from most people's model of how you
[11:07] do cross-disciplinary work.
[11:09] I think most people's model is like, you get a computer scientist and you get a
[11:13] doctor and you put them in a room together and then they'll produce something good.
[11:18] What doesn't really work about that model is that you put those two people
[11:22] in a room together and they often don't have anything to talk about.
[11:25] And in fact, they don't even really speak the same language.
[11:28] Um, and so the work that gets done is kind of, it has this superficial feeling
[11:33] because nobody can really get into the interesting part of either one field.
[11:38] So even though it seems ridiculously inefficient, I think that the right model
[11:43] is the doctor actually needs to learn the computer science and the statistics
[11:47] and the statistician or the computer sciences actually needs to make big
[11:50] investments in learning about health.
[11:53] And I think that's, even though again, it doesn't seem, it seems like you should
[11:57] be able to do it the other way and it's like, wow, this is really a lot of work
[12:01] for both people to learn both things.
[12:03] But I actually do think that that's how the best work that I've seen in this space
[12:07] and the work that I've done, that I'm most proud of, um, that's how that's happened.
[12:12] So that's a great transition to your work and in your research.
[12:15] So you've worked on a lot of different topics, uh, at the intersection of
[12:18] machine learning, economics and medicine.
[12:21] And I think one of the very impactful, uh, threads of work that you've led
[12:26] is on the topic of algorithmic bias.
[12:28] I thought we could start off with your 2019 science paper, which is titled
[12:33] Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.
[12:38] So this paper, to me, was really a very powerful and clinically important
[12:42] example of the importance of how you formulate a machine learning problem,
[12:46] how you choose what your task is, what your target is to predict, and what this
[12:51] encodes both about society and about the generalizability of a model that
[12:56] is trained to predict some outcome.
[12:58] So maybe we could just start off by you summarizing the paper, telling
[13:02] us what you set out to do, and maybe also you could share how this
[13:06] project got started, how you got interested in this particular problem.
[13:10] I think to set the stage for this work, one of the things that I think
[13:16] a lot of people listening to this will know is that health systems have made
[13:20] huge investments in population health.
[13:22] So I think the old model was, you know, you just treat one patient at a time.
[13:26] They come in, you treat them, you treat and release.
[13:29] And I think that as the healthcare system has changed, health systems have changed
[13:33] to a view where they're responsible for a population of patients and they're
[13:38] responsible for caring for those people.
[13:40] They're responsible for the costs of those people.
[13:42] They're just like, take charge for this big population of patients.
[13:45] And one of the things you need to do in that setting is you need to make
[13:49] sure that the people that you're taking care of, Don't get sick.
[13:53] So some of those people are going to get sick over the next year, and if
[13:57] you knew which patients were going to get sick, you could actually help
[14:00] them today, but you often don't know.
[14:02] And so there's a range of things you can do.
[14:05] High risk care management is the, the big category of things, but
[14:08] it's basically just extra attention.
[14:09] Like a nurse practitioner will come to your house, they will refill your
[14:14] medications, they'll examine your foot for ulcers, they'll, they'll
[14:17] do all sorts of things to prevent.
[14:19] You from ending up in the emergency department and seeing someone like me
[14:23] and getting admitted to the hospital for antibiotics or diuresis or whatever.
[14:27] So that's the kind of the big picture and, and you can see where algorithms could be
[14:30] super useful for this task because what you're trying to do is, okay, I see a
[14:34] bunch of people, I have a bunch of data on them, some of them are gonna get sick.
[14:37] I need to find these needles in this haystack so that I can
[14:40] get on top of their care today.
[14:43] And so this has become a huge industry.
[14:45] So we studied one commercial product.
[14:48] Um, it's an algorithm made by a analytics subsidiary of a large insurer.
[14:53] That product is used by their estimates to make these decisions
[14:57] about who gets access to extra care for about 70 million people every year.
[15:01] The family of algorithms that works just like the one we studied
[15:05] about 150 million people per year.
[15:07] So like the majority of the US population is being run
[15:09] through one of these algorithms.
[15:11] And it's not just if you go into the hospital, you can be sitting
[15:14] on your couch and these algorithms are churning through your data.
[15:17] Gating, whether you get access to extra help with your chronic conditions or not.
[15:22] So that's the the landscape we, because at the time I had a partnership with a
[15:28] academic health system in the Boston area.
[15:31] We had access to the algorithm risk scores from this one commercial product.
[15:37] And we also had access to this rich data set of what happened to people.
[15:41] And so the way this health system was using the algorithm was three times a
[15:44] year, they would just run the algorithm that would generate a list of high risk
[15:49] people and then a small fraction of the people would get fast tracked into the
[15:53] high risk care management program that gives them extra help with their health.
[15:56] The next 50% approximately would get shown to their doctor.
[16:00] So the doctor gets to decide, so there's a human in the loop deciding
[16:03] is the algorithm right or not?
[16:05] And then the bottom 50% just gets screened out so they don't get
[16:08] offered access to this program.
[16:09] So the score was really dictating whether you got access to help or not.
[16:14] And we were very interested in whether that score was biased.
[16:18] So one of the things that I learned from this project is that if you wanna study
[16:24] bias, you have to start by defining bias.
[16:27] Because even though, you know, I think in a lot of settings is very
[16:30] clear what bias looks like, especially in human settings where you see
[16:34] discrimination with algorithms, it's a lot harder to pin down what bias is.
[16:40] And so having a crisp definition of bias was really important for us to study the
[16:46] extent of bias, uh, in this algorithm.
[16:48] And so, given how that algorithm was being used, what we thought was okay,
[16:52] so the algorithm score is determining whether you get access to help or not.
[16:56] And so given two people with the same score, they should
[17:00] need the same amount of help.
[17:01] They should have the same health needs.
[17:04] And the color of their skin shouldn't matter.
[17:07] But unfortunately, that's not what we found.
[17:08] So what we found was that if we took those two hypothetical patients with
[17:12] the same score, and we followed them over the year after the algorithm
[17:16] made its prediction and generated that score, the black patients did
[17:19] far worse than the white patients.
[17:21] They had far more flare-ups of chronic conditions that landed them in the ER
[17:26] and in the hospital they had worse blood pressure, they had worse kidney function,
[17:29] they had worse hemoglobin A one C, like everything was worse, even though the
[17:33] algorithm was viewing them the exact same way and gating their, the, the same level
[17:38] of access to high risk care management and, and extra help with their health.
[17:43] So we had a precise statement of what bias would look like.
[17:48] The algorithm failed that test.
[17:50] And then the question is, well, why did it fail that test?
[17:52] And it failed for a very, you know, a deceptively simple reason related
[17:58] to how the algorithm was built.
[18:01] So, What we wanted to do in this setting was find people who are going
[18:05] to get sick, but in your dataset, there's no variable called sick.
[18:11] There's a bunch of data in there, but they're not one variable.
[18:14] So you need to figure out what exactly is my algorithm predicting what variable?
[18:19] And it's the variable that the people who built this algorithm, and a lot of other
[18:22] algorithms like it chose, is a variable called total medical expenditures costs.
[18:28] So it looked ahead to see, okay, starting from this point, how
[18:32] much is the healthcare system going to spend on a given person?
[18:35] And let's use that as a proxy for how much care they need.
[18:40] So how much care they got.
[18:42] Denominated in dollars was the proxy.
[18:45] How much care they needed was what we wanted.
[18:47] But as anyone who has tried to access the healthcare system in this
[18:51] country, but in many other countries knows needing healthcare and getting
[18:54] healthcare are not the same thing.
[18:57] And there are certain groups of people for whom that is particularly true.
[19:01] And those people are disproportionately non-white and they're disproportionately
[19:05] socioeconomically disadvantaged.
[19:07] And that was kind of a known fact, like we know that not everybody who
[19:11] needs healthcare gets healthcare, but translating that known fact into
[19:15] algorithm design, unfortunately didn't happen in this case or many other cases.
[19:19] And so we.
[19:21] Huh everyone.
[19:22] I, I think there's this tendency, everyone gets angry at the algorithms.
[19:25] It's like, oh, this algorithm is by, but the algorithm, the algorithm is
[19:28] doing exactly what we told it to do.
[19:30] It's not the algorithm's fault that costs are lower for black
[19:34] patients than for white patients at the exact same level of health.
[19:37] That's our fault.
[19:38] That's society's fault.
[19:39] But now the algorithm is mirroring that deep bias in society because
[19:44] we chose to have the algorithm mirror that deep bias in society.
[19:48] So that's where the bias came from.
[19:51] Do you know if the manufacturer tried to predict health state or
[19:55] need as opposed to cost before deploying this model in practice?
[20:00] So I don't think they did.
[20:03] And we actually, um, when we first got these results before we
[20:06] published, we struggled a little bit for what do we do with this result.
[20:11] We had some suspicion that this paper was gonna get attention.
[20:15] They ended up getting a lot more attention than we thought.
[20:18] But what we actually didn't want is for that company to learn
[20:22] about this when it was covered in a newspaper or, or something.
[20:26] And so we, because we actually just wanted to make this better, we wanted
[20:29] that company and, and all the other companies to actually change the way
[20:32] that we're building these algorithms.
[20:34] So we actually reached out through our networks to people at this
[20:38] company and we ended up connecting with the technical team that
[20:41] built and managed this algorithm.
[20:43] And what I came away with from that work is that, Nobody, not this team,
[20:48] but not anyone who was in this loop of people buying the algorithm, applying
[20:52] the algorithm to their patients.
[20:53] Nobody had thought about this problem.
[20:56] And we all use cost as a shorthand for need.
[21:00] Like we do it in policy circles, we do like it.
[21:02] This is a very, this is a very widespread error in health policy circles of
[21:09] using costs as a proxy for need.
[21:10] So I don't even think this was on anyone's radar screen, but when we
[21:13] pointed out the problem, they were really motivated to work with us to fix it.
[21:17] So we actually worked with that technical team to build an improved version of
[21:21] the algorithm that predicted a bunch of outcomes that were much more closely
[21:25] grounded in health than in cost.
[21:27] And we dramatically reduced the amount of bias and the algorithm
[21:30] that resulted from that process.
[21:32] And so my first.
[21:34] Reaction when I saw these results was, yeah, this kind of tracks, we say we're
[21:39] trying to take care of patients, but actually we're trying to reduce costs.
[21:42] And so of course the algorithm was predicting cost.
[21:45] That's what everybody cares about.
[21:46] But I actually don't think that's right either, because even if you just cared
[21:50] about reducing costs, You still wouldn't predict costs and target those people.
[21:56] There's a bunch of high cost people that you can't do anything about, and
[21:59] in fact, in a lot of these, like when you look at the lists of patients
[22:03] who score very high and need extra help, what kind of patients are they?
[22:07] Well, their patients with metastatic cancer, high risk care management
[22:11] is not gonna make a dent in the cost of metastatic cancer.
[22:14] That is just a cost we bear as a health system and as a society.
[22:18] People with.
[22:19] Total knee replacements, they cost a lot of money in that year, but
[22:22] they're not gonna keep costing money.
[22:24] And if they do, again, there's not.
[22:25] So we're, we're interested in preventable costs, not total cost.
[22:30] And so if you're interested in preventable costs, it's really not clear that
[22:33] you wanna be predicting total cost.
[22:34] You wanna be predicting these avoidable costs that come from emergency department
[22:38] visits that could have been handled over the phone, hospitalizations
[22:42] that could have been prevented, had someone's diuretic dose been increased
[22:46] in time, like things like that.
[22:48] So I think that process of going through the problem formulation, as you mentioned,
[22:53] it's super, super important, but it's often an afterthought when we build
[22:57] algorithms because, you know, often, frankly, because it's such a, a pain in
[23:01] the ass to get the data and clean it and do all this stuff, that by the time you
[23:04] have the data, you're like, all right, you know, let me just pick a variable and
[23:07] predict it because this has already taken six months and I'm way behind schedule.
[23:11] I think you're totally hitting on something that is, uh, so central, right?
[23:16] Which is that getting the data, accessing it, cleaning it up is often
[23:21] the bottleneck, the challenge, and the really big hill that we have to
[23:25] climb in medical machine learning.
[23:27] And I think the other thing you're, you know, you're pointing out there's a
[23:30] lot of attention on algorithms, right?
[23:32] But all this work beforehand that goes into it, that can actually
[23:35] take up most of your time.
[23:36] Um, so you mentioned that you contacted the manufacturer.
[23:40] And that you worked with a manufacturer to then change the way that this
[23:43] prediction was happening, right.
[23:45] To change the model itself.
[23:46] I'm, I'm really impressed.
[23:48] You know, there's sort of the technical side of this paper, but there's the social
[23:50] side of this paper as well, which is navigating that process to even get the
[23:54] risk scores, get the data, use it in this way, and then to work with a manufacturer.
[23:58] I think there's a lot of well-intentioned medical machine learning researchers
[24:02] who are gonna be listening, who are thinking about similar challenges
[24:05] where they're working with proprietary algorithms or technology.
[24:08] What advice do you have to them about how to navigate the social side of this
[24:12] process in addition to the technical one?
[24:15] I think first it's important to realize that it's such a luxury
[24:19] to have access to the scores.
[24:20] I mean, in, in criminal justice and in lending in a lot of other areas
[24:24] where people are trying to study algorithms and algorithmic bias,
[24:28] researchers just can't touch the scores.
[24:30] It's an amazing luxury to be working in this area in health where the
[24:35] scores are sitting in a data warehouse alongside all of the outcome data that
[24:39] you can use to validate the prediction.
[24:41] So it's a great area to be working in if you're coming at it from the health angle
[24:45] and it's, it's a lot easier than people who are trying to do this in other areas.
[24:50] I think what I learned from working, at least with the, the technical team at
[24:57] that company is that, and, and it's, it's hard to remember when you're studying
[25:03] something as awful and, and shocking as bias at, at a large scale, but.
[25:09] What I tried to remember and, and what was born out in all of my interactions
[25:13] was that most people actually really wanna do the right thing.
[25:17] Like we were on one call with this team, it was like early people hadn't joined,
[25:22] and this one person on the team said, I am so glad that you all found this problem.
[25:29] Um, I took this job instead of a job at a tech company because I actually
[25:34] wanted to make a difference and I wanted to work in health and I wanted
[25:37] to get people care that they needed.
[25:38] And this is the last thing I wanted to do.
[25:40] And it's so great that we have the chance to fix this thing.
[25:43] So, you know, it's possible these people deserve an Academy
[25:46] Award and they're actually evil geniuses and they got caught.
[25:48] But I don't think so.
[25:50] I think people mostly wanna do the right thing, and we're all
[25:54] learning about this together.
[25:55] This is a very new field, it's a new area.
[25:58] We don't have great understandings about how bias gets into algorithms, so we're
[26:02] all trying to figure this out together.
[26:05] And I think having that positive attitude at least made the work
[26:09] a lot more satisfying for me.
[26:10] And I think it was reciprocated in working with a lot of other people who
[26:14] wanna affect positive change as well.
[26:16] And I just would wanna say the, like in addition to highlighting these, how.
[26:21] Algorithms can operationalize structural biases on a huge scale.
[26:25] I think for me it also highlighted the urgency for really
[26:28] thinking about these things.
[26:29] 'cause we tend to talk about the impact of AI in the future tense.
[26:34] Will AI replace doctors?
[26:36] What will happen in the future?
[26:38] But as your work shows these simple forms of ai, if you wanna, you know,
[26:41] be generous with the definition of ai.
[26:43] Already touching hundreds of millions of lives.
[26:46] And so this isn't something that we can kick down the road and worry about later.
[26:48] We actually have this problem now and it's only gonna become more pressing over time.
[26:53] Yeah.
[26:53] Thanks for bringing that up, that it was very surprising to me how
[26:56] widespread these algorithms were on this backend side of the healthcare system.
[27:02] You know, I think you go into your doctor's office, you're sending
[27:05] faxes, you're filling out paper forms, you're like, what the hell?
[27:08] This is like, there's no AI going on here.
[27:10] And that's right.
[27:11] Largely on the clinical side, but on this population health management side,
[27:16] on a lot of other operational sides like clinic bookings, like things that have
[27:20] a direct impact on health, these tools are already in very, very wide use.
[27:25] We a after, because we got attention for that study, we got a lot of inbounds
[27:30] from health systems, tech companies, a lot of others that were interested in
[27:35] evaluating their algorithms for bias.
[27:36] So we did a lot of follow up work and case studies and efforts to diagnose
[27:40] and fix bias and other algorithms.
[27:42] And these things are.
[27:43] Everywhere and everywhere we looked almost we found bias in the
[27:47] algorithms that were being used.
[27:49] So I think that's a great transition point because the next paper that we
[27:52] wanna talk about kind of flips that on its head and talks about how actually
[27:56] machine learning may be an opportunity to reduce healthcare disparities.
[27:59] So I'd like to talk about your paper, an algorithmic approach to reducing
[28:02] unexplained pain disparities in underserved populations where you go
[28:06] after longstanding pain disparities between different populations and
[28:11] actually show how algorithms can find things that explain sort of
[28:15] self-reported versions of pain.
[28:16] So could you, could you set that up for us and help us walk through
[28:19] that and, and, and was this a.
[28:21] Actually a continuation of the paper that we just talked about or, or like
[28:25] sort of what was the genesis of this one?
[28:27] So this paper was led by my colleague Emma Pearson, who's a fantastic computer
[28:32] scientist at Cornell and, and she incidentally is a great example of
[28:38] a computer scientist who's made huge investments, not just in learning about
[28:41] knee x-rays and MRIs and everything she had to do for that paper, but also
[28:45] learning about a ton of other areas where she's applied algorithms from covid to
[28:50] criminal justice to lots of other things.
[28:52] So I think she's a really shining example of the kind of
[28:55] cross-disciplinary expertise that we were talking about earlier.
[28:59] So this paper actually came out of a bet.
[29:03] I was at a conference and my co-author and friend David Cutler, who's a
[29:08] health economist, was presenting some research that he was doing on pain
[29:13] and he was looking specifically at knee osteoarthritis, which is one
[29:17] of the most common causes of pain.
[29:18] And he was resenting data that.
[29:20] Showed these very striking disparities in pain that exist across race
[29:27] and socioeconomic gradients.
[29:29] And so one of the many injustices in ours I is not just socioeconomic injustice,
[29:35] but if you think about the injustice that comes from the fact that some people
[29:39] experience pain daily, much more than others, that's just like another crazy and
[29:43] visceral kind of injustice that is largely missing from like public statistics.
[29:49] So we just like don't capture data on pain, but it's really shocking if
[29:52] you just think about what that means for humans on a day-to-day level.
[29:56] Okay.
[29:57] So the research that David was presenting is a great example of a pretty consistent
[30:02] finding in the literature, which is that not only do, for example, black patients
[30:06] have more pain than white patients, but that difference, that gap in pain
[30:10] persists even when you control for.
[30:14] Severity of disease in this case, in the knee.
[30:17] So what you do is you take two patients, one black, one white, and
[30:21] you get two patients whose x-rays their knee x-rays were graded in exactly
[30:26] the same Calgren Lawrence score.
[30:29] Both of 'em have the same score.
[30:30] That's like the system that radiologists used to grade arthritis.
[30:33] You look at all the compartments of the knee, you assign each a score, you sum
[30:36] it up, it goes from zero to four or five.
[30:39] Anyway.
[30:39] I'm not a radiologist, uh, far from it.
[30:41] So when you do that, you still find that black patients report
[30:45] more pain and that's interesting.
[30:48] And so I think David's interpretation, and I think the interpretation that
[30:53] most people have is that there are psychosomatic and psychiatric stressors.
[30:58] There are differences in medical care.
[30:59] There's lots of stuff that's going on outside the knee that could be
[31:02] contributing to this, uh, and causing this gap because we've held the knee constant.
[31:08] And so after the presentation, I went up to David and I was
[31:10] like, I bet it's in the knee.
[31:12] He is like, no, it's not in the knee because we've controlled for that.
[31:14] And I was like, no, I, I bet it's still in the knee.
[31:17] So that was the, the friendly bet that led to, um, what were the stakes?
[31:22] I neither David nor I is an actual betting man, but you know, when your
[31:26] academic a gentleman's bet, then when your academic credibility is
[31:29] on the line, Andrew, it's, you know, it's, it's worth, it's priceless.
[31:34] So we were able to get data from this n I H study that David had been using, but
[31:40] not only did we get the data on pain and race and, and Kelgren Lawrence grade that
[31:44] data had gotten, we also got the x-rays.
[31:47] And so what we did is we, we trained an algorithm to look at the x-ray.
[31:51] And predict whether a patient would say, this knee hurts or this knee doesn't hurt.
[31:57] And it's worth drawing a distinction between that approach and what I
[32:02] think most papers in literature do, which is like, well, how do you train
[32:05] an algorithm to read a knee x-ray?
[32:08] You train it to replicate what a doctor would say.
[32:11] Mm-hmm.
[32:11] About that knee x-ray.
[32:13] So you'd input the x-ray and you'd ask it to output the Kelgren Lawrence grade, but.
[32:18] If you're me and your hypothesis is, well, I think radiologists are just missing
[32:22] stuff in the knee that's causing pain.
[32:25] That's exactly what you don't wanna do.
[32:26] So you don't want the algorithm to replicate the limitations of human
[32:29] knowledge and the errors built into it.
[32:32] You want it to actually find new signal.
[32:34] So we trained it to listen to the patient instead of replicating
[32:38] what the doctor would've said.
[32:40] And so that gave us an alternative measure, an alternative fact,
[32:44] if you will, about the knee.
[32:46] So radiologists have their score, they have a Kelgren
[32:48] Lawrence grade for the knee.
[32:50] We have our algorithmic pain score, which is capturing the predicted amount of pain
[32:55] from the pixels that we see in the knee.
[32:58] And what we found is that if we then go back to that pain gap between black
[33:02] and white patients, there's some gap.
[33:04] So black patients are in much more pain.
[33:06] If we control for the radiologist's interpretation of that pain, we
[33:10] account for I think 9% of that gap.
[33:13] But if we control for the algorithm's interpretation of that x-ray, We
[33:17] account for about half of that gap.
[33:19] So the algorithm is doing a much, much better job of explaining pain
[33:22] overall, but it's doing a particularly good job of explaining the particular
[33:27] pain that radiologists miss and that black patients report, but that can be
[33:32] traced back to some pixels in the knee.
[33:36] Um, yeah, I, so I, you know, putting all my stats at like models
[33:40] are meant to explain variation.
[33:41] That's fundamentally like what a model does.
[33:43] And I think what you're saying here is that, that the deep learning
[33:47] model actually does a better job at explaining subjective pain
[33:50] variation than the human model does.
[33:52] Is that sort of a fair condensation?
[33:55] That's exactly right.
[33:55] If you look at the R squared of pain, so like of all of the variation in
[33:59] pain, how much does the model explain versus the radiologist's interpretation?
[34:05] The model, I think was about 50% more overall.
[34:08] Wow.
[34:09] Wow.
[34:09] Now again, it was still low.
[34:11] Yeah.
[34:12] And there, there's still a lot going on in pain that cannot be traced to
[34:15] the pixels, but when you're looking at the pixels, you wanna get the
[34:19] best explanation of pain as possible.
[34:21] Yeah.
[34:22] And again, this I think, really highlights what I like so much about
[34:25] your work is that all technologies have this duality aspect to it.
[34:29] They're sort of, uh, net neutral.
[34:31] They can be harmful or good depending on the context they're
[34:33] used, the data they're trained on.
[34:35] And I think you sort of have shown the dynamic range of possibilities
[34:38] for good versus ill with how we think about medical machine learning.
[34:43] Thanks for pointing that out.
[34:43] I could not agree more with that statement.
[34:45] I think they're just tools and the tool is neither good nor bad.
[34:50] It's just about how we build the tool and how we choose to apply it.
[34:55] And that can do terrible things or it can do really amazing things
[34:58] and it's really just all up to us.
[35:01] Mm-hmm.
[35:01] Yep.
[35:02] Okay.
[35:03] So Raj, are you think we're ready to transition to the lightning ground?
[35:07] Should we ask the ad about the data sharing and AI innovation?
[35:10] Oh, right, yes.
[35:11] Let's do that.
[35:11] Yeah.
[35:11] So we've done a good job at exploring a lot of these early topics, but we do
[35:14] wanna give you a chance to talk about Nightingale, some things like that.
[35:18] So I'll ask that one.
[35:19] So in addition to your, your primary research, um, you've also taken some
[35:24] entrepreneurial roles around building companies and platforms that can help
[35:29] others do, uh, sort of what you've done in the past by getting access to these rich
[35:34] data sets to answer important questions.
[35:35] And I think importantly, the question for these data sets is not, not
[35:39] necessarily pre-specified, it's kind of more like a, a playground or a sandbox
[35:43] or just sort of a place to do research.
[35:44] So could you walk us through Nightingale and sort of, um, how that started and sort
[35:49] of what you hope to achieve with that?
[35:51] Yeah, I think to, to go back to an earlier part of our conversation,
[35:56] Getting the data that you need to do your research is a huge, huge preoccupation
[36:03] of any researcher in this area.
[36:05] And I think the problem is that the data are essentially locked up inside of the
[36:12] health systems that produce the data.
[36:14] And it can be really perverse.
[36:16] Like I, I'm sure for example, that if you are appointed at the medical school at
[36:20] Harvard, but because the medical school is a different administrative entity to
[36:25] the Brigham Women's Hospital or the Beth Israel Deaconess Hospital, literally
[36:28] across the street, you do not necessarily have a right to access that data, except
[36:33] if you go through some other channel's, not, we don't like people to know this
[36:37] until they agree to be a postdoc with us.
[36:39] So you're giving away all the, all the bad secrets.
[36:41] I'm sure you can edit, we edit this out later.
[36:43] Yeah.
[36:45] So, you know, the, so these things are really pervert and if you wanna
[36:47] hire an analyst to, to do anything with the data, you have to hire them
[36:51] through the hospital, which means you have to have a grant through the ho.
[36:53] So, It, it's Byzantine and it's very frustrating.
[36:57] And, and I think it's really holding back this space because
[37:02] it is multidisciplinary work.
[37:04] You need computer scientists, um, hospitals can't hire all the computer
[37:09] scientists that they would need to do this work, and they're not gonna win the
[37:13] war for talent, um, against Google or Facebook or, or even just computer science
[37:18] departments of different universities.
[37:20] It's just, you know, it is just a fact.
[37:22] So doing this work is just, you know, there are so many frictions and it's
[37:27] why I have a few papers that have, like, one paper took me seven years to write.
[37:32] Um, there's another one where we're at, I think year eight now.
[37:36] And like, you know, a lot of it is because I'm inefficient and an academic
[37:39] and just, I procrastinate a lot and I'm not that great at getting things done.
[37:43] But it's not all the blame.
[37:45] Some of the blame is on data use agreements and I r b
[37:49] amendments and de-identification problems and things like that.
[37:52] So a few years ago, uh, thanks to initially support from Eric Schmidt's
[37:57] Foundation, uh, Schmidt Futures, but then with follow on funding from the
[38:00] Moore Foundation, Ken Griffin, and a bunch of other donors, we set up a
[38:04] nonprofit called Nightingale Open Science.
[38:07] And what Nightingale does is we take that philanthropic money and we deploy
[38:11] it to build interesting data sets with partners inside the health system.
[38:15] So, uh, working with, uh, researchers at Providence, um, St.
[38:20] Joseph's a big health system all up and down the West coast.
[38:22] We bought two slide scanners and hired some pathology technicians, and we went to
[38:29] all of their breast cancer biopsy slides.
[38:32] These things were literally collecting dust on the shelf in a basement, and we
[38:36] just started digitizing those slides, linking them to electronic health
[38:39] record data, to cancer registry data to social security data on mortality.
[38:44] And if you go to nightingale science.org, you'll be able to fill out a very
[38:51] lightweight data use agreement and within a day or two, get access to those
[38:56] de-identified data and on an A W Ss platform with a Jupyter Notebook where
[39:00] you can just start doing your research.
[39:03] And we're exploring models and as you and I have talked about before, Andrew,
[39:06] of like how to get those data also into the ecosystems of researchers at
[39:12] universities who already have clusters who don't wanna do things on a W Ss.
[39:15] But we've actually had a lot of traction from researchers around the world who
[39:18] don't have their own compute clusters, who are delighted to, to use the a
[39:22] w s platform that we have things on.
[39:23] So, so we build up those data sets around interesting medical problems,
[39:27] in this case cancer metastasis.
[39:29] We de-identify the data and then we make those data available in a
[39:33] secure and we think ethical way on this platform where researchers can
[39:37] use it to do non-profit research.
[39:39] Um, so that's Nightingale.
[39:41] This is something that I think we've talked about before, but other than
[39:44] a charm offensive, which I know coming from Ed is a considerable
[39:48] force, how do you get the hospitals to actually, what is in it for them?
[39:52] Like why would they want to contribute to data from this?
[39:54] I know, you know, especially Boston area hospitals are notoriously stingy with
[39:59] their data, even with Harvard researchers.
[40:01] So how, what's the key to unlock that data?
[40:06] I think the key is that there are a lot of hospitals and so even in our
[40:10] case, if 200 say no, five say Yes.
[40:14] And so we were very lucky to find fantastic partners at hospital
[40:18] systems from big non-academic systems to small county hospital systems.
[40:23] And there were people who just, they had a problem that we could help
[40:26] solve, whether it was for research or something operational, and they
[40:29] were able to push that forward.
[40:32] And so we, we were very, very lucky to find the partners that we found.
[40:35] But you know, your question is such a good one because getting a hit rate of five out
[40:41] of 200 or so that we talked to, it was a frustrating process and it's definitely.
[40:46] Puts a limit on the scale.
[40:48] And so it was actually in the course of getting rejected, uh, hundreds of times
[40:53] that actually, like, there was another idea that occurred to us at the time
[40:56] and us as, uh, my co-author, sandal Mohan, who's an economist and I and a
[41:01] few others, is that the same problem?
[41:04] That was, I think a huge bottleneck for research was also a bottleneck for
[41:08] people who wanted to build products that would have a much more direct
[41:13] way of getting into the clinic than a paper that we as researchers publish.
[41:17] And so there's a for-profit that I also co-founded called Dandelion.
[41:22] And Dandelion has agreements with a handful of large non-academic health
[41:26] systems where we can curate and create data sets and build those data
[41:32] sets to serve the needs of startups.
[41:34] Life sciences company, medical device companies that wanna build AI products
[41:38] with a view to deploying them into the clinic for patient benefit.
[41:41] So do you act as like a, a broker in between the, the industry
[41:46] partner and the hospital system and.
[41:48] Yeah, that's exactly right.
[41:49] So I think the current way, if you're a an AI startup and you wanna get
[41:53] access to data to build a product, you have to build a bilateral relationship
[41:58] with a hospital and go through a very onerous contracting process.
[42:01] And, and so I actually know of a handful of startups that ran out of
[42:05] their Series A funding while they were waiting for these kinds of agreements
[42:09] to get finalized through the hospital.
[42:10] So it's really definitely stifling a lot of innovation that I
[42:14] think could benefit patients.
[42:15] And so the idea of dandelion is to reduce those frictions to make it easy for people
[42:21] who wanna develop products to access data, again, in a secure and ethical way.
[42:25] And I think the ethical, the ethical way, I think is an important one.
[42:28] 'cause I think you, you have to think hard about, well, we're selling data basically.
[42:34] And I think the way I've been thinking about it at least is there's a huge
[42:40] opportunity cost to leaving those data.
[42:43] Not used.
[42:44] There are so many insights and products that could directly benefit patients
[42:48] that are not getting developed today because it's so hard to access those data.
[42:52] So even though there are clear threats to privacy that we need to take
[42:55] into account and security, there's also this other risk that I think we
[42:59] spend a lot less time thinking about.
[43:01] But that is at least as large as the privacy risk, which is what?
[43:05] What about all of the things that could diagnose cancer
[43:09] early that we don't have today?
[43:11] Because nobody can develop the product.
[43:14] What about all of the ways we could do better triage in the emergency department
[43:18] and get care to people who need it faster and reduce the cost of care?
[43:22] Like what about all of those products?
[43:23] Where are they?
[43:25] Well, I think they're not being developed because it's so hard to get the data
[43:27] to develop the products to begin with, and that that's a problem too.
[43:31] Yeah, I, I agree.
[43:32] And the healthcare system is just very bad at trade-offs in the sense of weighing
[43:37] explicit costs versus implicit costs.
[43:40] And I think like they're very good at minimizing the, the risk of explicit harm.
[43:45] Um, but there's no accounting for the implicit harm done by lack of
[43:48] progress and lack of innovation.
[43:49] And I think that, you know, hanging out with economists is a great
[43:52] way to get into that mindset.
[43:54] And if there's a way that we can reduce that friction, I think that it'll
[43:57] hopefully tip that balance in a, in sort of a more favorable direction.
[44:01] Yeah, very well put.
[44:02] Is, yeah.
[44:03] On Dandelion and Nightingale, how do you think of what is sort of a for-profit
[44:08] activity versus a non-profit activity?
[44:10] Because in a way they're both data platforms accelerating AI in medicine,
[44:15] so how do you partition work or projects for one versus the other?
[44:19] Yeah, F fantastic question.
[44:21] I think the way I think about it is just to work back from the outputs.
[44:25] And so I think there's a lot of research that needs to be done
[44:28] to produce ideas and papers.
[44:31] Like there's a lot we don't know about what cancer is, which cancer
[44:35] is spread, like can we predict which cancer like, so there's just some
[44:40] basic science questions that we need answers to before we can develop.
[44:45] Products.
[44:46] On the other hand, there's some very practical things that we can start doing
[44:50] today, even without fully understanding the nature of cancer and metastasis to
[44:55] improve the care of patients with cancer.
[44:57] And so there's also a bunch of things we can do with data that have an
[45:02] outlet to things that benefit patients directly on non-academic timescales
[45:07] and with non-academic skill sets.
[45:09] And I think one of the things that I've really come to appreciate about the
[45:13] private sector and basically my new non-academic friends and acquaintances
[45:19] is, boy do they get shit done.
[45:21] They don't have projects like I have that have gone on for eight years.
[45:24] If it goes on for eight days, it's like, what's going on?
[45:27] What's taking so long?
[45:27] So there's an impatience and a raw competence that I've been trying
[45:33] to learn from, from that world.
[45:35] So separation of timescales is a, is maybe one, one principle to think about it.
[45:40] And then I, I have to ask what is behind the name Dandelion.
[45:44] I'm thinking about being on the playground and blowing the flowers and
[45:47] the little petals going everywhere.
[45:49] Uh, I think that's, that's definitely one of the images we were going,
[45:52] it, it's, you know, there's a dissemination kind of thing.
[45:55] It's a positive thing.
[45:56] Dandelions are pretty.
[45:57] Elliot, our c e o and, and co-founder, he'll say all that stuff, but
[46:02] then he'll admit that it's because his dog likes to eat dandelions.
[46:06] Maybe I'm misremembering my horticulture, but that's actually
[46:08] the way that they reproduce, right?
[46:10] That those things blow and then they Exactly, they planted
[46:13] and create new dandelions.
[46:14] So absolutely a very APTT metaphor.
[46:17] Okay, so now I think we actually are ready for the lightning round.
[46:19] So, um, buckle up, um, and, uh, we'll let you know, uh, if you
[46:24] win at the end of the round.
[46:34] So we're speaking today on March 14th, 2023, which is the day that Gvt four
[46:40] was announced and is now a thing.
[46:42] So we ask.
[46:42] Cool.
[46:42] It's also pie day.
[46:43] I thought this, I thought this was gonna be a pie day joke.
[46:46] Yeah.
[46:46] So, so, uh, that's actually true also.
[46:49] So maybe we will have to, we'll have to work into some, uh, some pie
[46:52] questions at the end of the round.
[46:54] So I guess the first question is, given everything that's happened over the last
[46:59] six months and given your work thinking about the duality of machine learning and
[47:04] medicine, I think it's fair to say that we're on the cusp of large language models
[47:09] starting to affect clinical practice.
[47:11] So I would like your wager on if M's.
[47:14] Large language models will be net positive for medicine over the next five years.
[47:19] And this is the lightning ground.
[47:21] So I'm gonna ask you to be brief in your responses.
[47:24] Definitely net positive.
[47:26] I would bet that it'll be more positive on the backend in terms of summarizing
[47:31] and abstracting information than it will ever be on the front end,
[47:35] given everything we know about the problems with the front end ette.
[47:40] I love that I get to ask this question.
[47:42] I think I know maybe one of the answers to this, and it might involve
[47:46] a beach, so I'm priming you a little bit here, but if you weren't in
[47:50] medicine, what job would you be doing?
[47:54] Um, you could dream, you could dream big.
[47:56] Yeah.
[47:57] I think there, there's a couple of things that I love.
[48:02] I, the real answer is that I get to keep being a doctor.
[48:05] I also get to do research.
[48:07] I also get to do this private sector stuff.
[48:09] So I feel like I, I am getting a lot of the best of many worlds.
[48:13] The other things that I would love to spend time doing are making espresso.
[48:17] Uh, I spent a lot of time doing that in my off time anyway.
[48:21] I recently learned how to surf, and so being a, a surf
[48:25] bum would also be really fun.
[48:27] And I think those are my two big hobbies for the moment.
[48:31] Awesome.
[48:31] Uh, the, the barista beach bum.
[48:33] I, I like that.
[48:34] Uh, counterfactual.
[48:36] Um, uh, so our, our listeners can't see this, but you're sitting in front
[48:40] of an impressive display of books.
[48:42] So this question is, what is your favorite book And maybe keeping
[48:46] with the beach bum theme, if you're, what's your Desert Island book?
[48:49] If you were stuck on a desert island, what book would you take with you?
[48:54] What a great question.
[48:55] I think, you know, how can one pick one's favorite of anything?
[49:00] I think the book that I've probably reread the most and so
[49:05] would be my Desert Island book.
[49:07] It's a book called The Rise and Fall of Modern Medicine,
[49:12] and it's an amazing history of.
[49:16] Medicine through the lens of the amazing discoveries that have been
[49:20] made by hobbyists, people doing frankly dangerous and irresponsible stuff.
[49:27] And it is just full of amazing stories.
[49:29] And who wrote that book?
[49:31] Uh, James Lanu.
[49:33] Okay.
[49:33] It's a, I think a general practitioner in London, if I'm not mistaken,
[49:37] and a fantastic historian, will AI and medicine be driven more by
[49:42] computer scientists or by clinicians?
[49:46] Clinicians with deep investments and expertise in computer science, statistics,
[49:53] economics, and behavioral science.
[49:57] So we're getting back to this multiple skill sets and domains of knowledge
[50:00] existing within the same mind as opposed to collaboration across separate silos.
[50:06] Yeah, and, and I should caveat that unfortunately by saying that those
[50:08] people are not being churned out at a very rapid pace by our system of
[50:12] medical education, unfortunately.
[50:15] Given your status as gentleman scholar and well-read philosophy student,
[50:20] if you could live in one other historical period, what would it be?
[50:26] I think the smart money is always on the future and so I'm gonna
[50:30] say maybe 150 years in the future.
[50:35] So it's like stuff is better, technology is better, but it's not so different
[50:39] that I would just be totally lost.
[50:42] Uh, is that conditioned on there still being a future in 100 years?
[50:45] So existential risk being roughly constant or diminishing over that period,
[50:49] you're taking that as for granted?
[50:51] Yeah.
[50:51] I mean, who, who knows?
[50:52] Maybe, maybe we'll be on Mars, but yeah, I, I think I would,
[50:56] given my probability estimate of that, I'd take it unconditionally.
[51:00] Um, got it.
[51:01] Alright.
[51:02] Last lightning round question.
[51:03] Do you think things created by AI can be considered art?
[51:08] Absolutely.
[51:09] Yeah, I, I think, I think all of these questions about is it sentient?
[51:15] Does it pass the Turing test?
[51:17] It's so beside the point to focus on the AI rather than to focus on
[51:23] the reaction that we have to the AI and what we can learn from the ai.
[51:28] So I think it can absolutely be art.
[51:30] I think it can absolutely teach us things, and I think it can absolutely
[51:33] do all those things without being conscious or, or sentient or whatever.
[51:37] It's really, I mean, not to sound narcissistic, but it's all about us.
[51:40] It's all about the humans, how we're using ai, how we are getting better
[51:44] and making society better with ai, the beauty is in the eye of the beholder.
[51:50] Um, so congratulations.
[51:52] I'm happy to report that you passed the lightning round with flying colors.
[51:56] I know that was a real, uh, a nailbiter there.
[51:59] What would've happened to me if I hadn't passed?
[52:01] Andrew?
[52:01] Uh, we are not allowed to disclose that.
[52:03] Oh my gosh.
[52:04] Okay.
[52:04] If you read carefully, if you read carefully in the material
[52:07] that we sent over that you had to sign, uh, it was included in that.
[52:12] Okay.
[52:12] So we'd like to just have some big picture, I think topics that we'd like
[52:16] to discuss with you before we let you go.
[52:18] And again, this goes back to all of the work that we've discussed so
[52:21] far, and I think it's less about AI and more about us, like you said.
[52:27] Do you think that AI over the long term is going to reinforce
[52:32] or reduce healthcare disparities?
[52:34] And that can again, be a, a societal question.
[52:37] Are we gonna get our stuff together and figure this out?
[52:40] Or if not, which way will AI tip the scales in that scenario?
[52:45] I think from the point of view of.
[52:48] Today I think we often overvalue or overestimate big historical secular
[52:57] trends and undervalue personal agency and the fact that what we do matters.
[53:04] And so I think that question depends entirely on we're, we're at a very
[53:09] critical moment now, and we don't have a lot of regulation right now.
[53:12] We don't have a lot of practices that inform how we adopt or purchase or scale
[53:17] up AI in lots of different sectors.
[53:19] So a lot of things are gonna get decided now, and a lot depends on the current
[53:24] generation of researchers, clinicians, health system executives, regulators
[53:29] at the F D A and other agencies.
[53:32] Like what we do now is really gonna matter for that long-term view.
[53:36] And I think it's critical that we get it right now.
[53:39] And so all that said, I.
[53:41] If you look at the track record of most technologies, they tend
[53:47] to reduce disparities over time.
[53:49] And so I think, you know, my base rate estimate for whether technology
[53:53] increases or decreases disparities is that it decreases disparities
[53:57] as all of the innovations in our healthcare system have done over time.
[54:02] I think that's a very reasonable stance.
[54:04] I guess I can just be pessimistic that structures that have been in place for
[54:09] so long are gonna get solved before.
[54:12] 'cause I, I do feel like there, there has to be sort of an order of operations
[54:15] there, there's a sequential dependency that if those structural problems
[54:19] don't get solved before AI becomes entrenched, then you know, as your
[54:23] work and others have shown, then it can just operationalize all of that.
[54:26] So, but no question.
[54:28] I agree that sort of the longer term view of that, um, as trending towards,
[54:32] towards balance, I think there's, there's another thing that I've started.
[54:36] Becoming conscious of and trying to work towards, which is using
[54:41] the technology to actually fight against those structural problems.
[54:46] And I think it's hard to see sometimes because if you look
[54:49] at something like wearables Yes.
[54:51] Who, who has wearables right now?
[54:53] A bunch of rich people.
[54:55] And so right now, if you look at the impact of wearables, it's like, yeah,
[54:59] it's not, it doesn't look great.
[55:01] But if you think, if you step back and you think about wearables as a tool to
[55:06] fight inequality, well it would be great if we could cut out all of the problems
[55:12] with accessing the medical system.
[55:14] Through the use of something that is in everyone's pocket, you know,
[55:18] like on your cell phone or on your wrist, and cost keeps going down.
[55:22] So I think that's not a crazy thing to think.
[55:24] So it would be fantastic if you could get access to the quality of
[55:28] medical advice that rich people have access to on your phone tailored
[55:32] to you and your heart rate and your blood pressure and things like that.
[55:35] So I think these technologies, to pick up a, a recurring theme of this conversation,
[55:40] they can be used, their dual purpose.
[55:42] They can be used to reinforce disparities or they can be used to fight against them.
[55:46] And I think that given how entrenched the current structures are, using technologies
[55:52] to disrupt some of those things and get access to people who don't have
[55:55] access is, is a very reasonable goal.
[55:59] Soya.
[55:59] I think another one of the big recurring themes of this conversation to me is
[56:03] that where really high impact, high quality research tends to happen, or at
[56:09] least one of the areas, uh, one of the sort of set of circumstances where it
[56:12] tends to happen is when multiple skills exist within the same person, right?
[56:17] So someone who's a computer scientist, who has a deep appreciation of medicine
[56:21] or economics and can speak both languages and can be creative about how they
[56:25] see connections across these fields.
[56:27] So we have a lot of early career clinicians who are listening to this
[56:30] podcast, med students, residents, and they are engaged in medical school, right?
[56:35] They're in medical school or residency or training right now.
[56:39] What do you think they should know about AI to help them
[56:42] prepare for a career in medicine?
[56:44] Uh, what could they maybe self-learn?
[56:46] Who can they talk to?
[56:47] What could they do at that point in their training journey to really be able
[56:51] to, uh, to speak to, to both skills?
[56:54] I think, you know, that.
[56:57] There are a few things that worked well for me that I'll just restate.
[57:01] I don't know if this is optimal.
[57:02] Uh, it's almost certainly not, but it worked for me.
[57:05] So I think the first thing I would probably suggest is that
[57:11] AI is a subset of statistics.
[57:15] It's an applied version of statistics with real data sets and, but there
[57:21] is no substitute for learning the basic statistical stuff.
[57:24] And I think as a starting point, that is an amazing place to start to get a handle
[57:28] on thinking about how AI works, where to apply it, where it can go wrong, all
[57:33] of all of those really important things.
[57:35] We're blessed to live in an age where you can just, Go online and take a class
[57:43] by some of the best statistics teachers in the world, and he's just like, do it.
[57:48] Instead of watching a bad TV show, it was about the same amount of time.
[57:52] So I think that's an amazing resource to take advantage of.
[57:56] Now, Khan Academy is great, like there are so many great resources for
[58:00] learning things online, but starting with statistics or in my case, for
[58:04] me, the toolkit that always made the most sense or resonated with
[58:08] me the most was the economics, the microeconomic toolkit for dealing with
[58:13] data that's produced by humans and is messy and error prone and and driven
[58:18] by incentives and things like that.
[58:19] That seems a lot like medicine to me.
[58:21] And that way of thinking about data and analysis was
[58:25] always very compelling for me.
[58:26] So there's a book called Mostly Harmless Econometrics that I really liked.
[58:31] Uh, there's another version of that same kind of curriculum
[58:34] called Mastering Metrics, which is in some ways even better.
[58:37] But I, I think those are two fantastic books.
[58:40] Um, so would you be in favor of the U S M L E adding a block on
[58:44] econometrics to, uh, to licensing?
[58:46] You know, I think the question with these things is like, you can't just
[58:50] add, you have to cut, so what do you cut?
[58:51] Right.
[58:52] I think there's a lot to cut, but yeah, I think absolutely.
[58:55] I think being a good doctor over the next 20, 30 years, it is just hard to
[58:59] imagine you can do that without knowing how to evaluate claims about vaccines
[59:05] and masks and a lot of other things that have come up, you know, a lot over
[59:08] the past couple of days, like medical knowledge is constantly evolving.
[59:11] It'll evolve even faster as AI tools come into play.
[59:15] Being able to critically evaluate those claims is just incredibly important.
[59:18] So evidence, appraisal and statistics as sort of the foundation for
[59:23] the house, uh, yeah, absolutely.
[59:24] Before you absolutely delve into it.
[59:25] Yeah.
[59:25] Yeah.
[59:26] And just thinking with data, I.
[59:27] But it's just like, it, it's, it's related to your earlier problem formulation
[59:31] point where it's just like thinking about how to take an abstract question
[59:37] and then think about what is the data frame that would answer this question.
[59:41] That is just like an incredibly valuable skill.
[59:44] I'll also say that one, to go back to chat, G P T, I don't know if you all
[59:48] have been doing this too, but I've been using it to write code a lot and it's
[59:54] actually like dramatically increased the amount of code that I write.
[59:58] So I, you know, I have some research assistants, some PhD students and you
[1:00:02] know, because I'm, I don't code that much anymore, it's like often easier for me to
[1:00:07] just ask someone to do something for me, but it's like a little unsatisfying, like
[1:00:11] I don't, like I know what I need done.
[1:00:14] I just like remembering which libraries to load and how to specify
[1:00:18] the stupid reshape command and all these, it is just like so painful.
[1:00:22] But now I can just type it in English.
[1:00:24] And then it generates a code snippet and then I can paste that in and
[1:00:27] like 80% of the time it works.
[1:00:30] It's incredible.
[1:00:31] So I actually think that for a clinician who has access to some data and
[1:00:36] knows some basic, like knows what are the statistical operations that she
[1:00:40] needs to do, this is an amazing tool.
[1:00:43] It's an incredible tool that I think for me, it's gonna honestly reshape a lot of
[1:00:48] how I do work over the next few years.
[1:00:51] Yeah.
[1:00:51] Andre Cari noted AI researcher and now at Open AI had a quote that was
[1:00:56] like the number one fastest growing programming language is English.
[1:01:00] In the wake of ChatGPT love.
[1:01:01] So I think love, a lot of people love are having a similar experience to you.
[1:01:05] Yeah.
[1:01:05] Speaking of things that people waste time on, uh, I spent too much time
[1:01:08] on Twitter arguing whether or not AI is a proper subset of statistics.
[1:01:13] Oh God, I shouldn't have wandered until this.
[1:01:15] I will not take the bait this time.
[1:01:16] I've learned my lesson.
[1:01:18] Change topic.
[1:01:19] Change topic immediately.
[1:01:20] Yeah.
[1:01:20] Well, maybe I, like, here's a, something that I, I heard recently that I like,
[1:01:25] which is that it's as engineering is to physics, data sciences to statistics.
[1:01:33] How do you feel about that?
[1:01:34] Um, analogy?
[1:01:36] Yeah.
[1:01:36] I, I, I find that unobjectionable and, uh, I think that I would just, there
[1:01:41] are parts of AI that have nothing to do with statistics is all I would say.
[1:01:44] I, I totally agree with that too, but, uh, impedance argument at best.
[1:01:48] So one of the last questions for me, which I think that you're gonna have probably
[1:01:53] a few things to choose from for this one, but what is your most heterodox opinion?
[1:01:58] Um, with respect to accepted wisdom in, in the field.
[1:02:04] I mean, you know, uh, how, how much time do we have?
[1:02:09] Like, I can, we can talk about essential oils.
[1:02:12] Yeah.
[1:02:12] Or you know, um, I think, you know, maybe, I think it's, it's probably
[1:02:19] related to medicine as a field.
[1:02:23] And I think my view today is that medicine is not its own field of study.
[1:02:32] And I think that that's hard to argue with that on, on some literal level.
[1:02:36] Like you can't get a PhD in medicine.
[1:02:38] And I think that it's because the data that make up medicine, like the things
[1:02:45] that get generated about patients in the electronic health record, in
[1:02:49] radiology PAC systems, they're very hard to handle with traditional statistics.
[1:02:57] And so I think the field is stuck between a bunch of rules of thumb
[1:03:04] and kind of like an arts and crafts sort of approach to, to doing things
[1:03:09] where it's like maybe more charitably.
[1:03:11] Artisanal.
[1:03:12] Yes.
[1:03:13] It's it.
[1:03:13] Thank you.
[1:03:14] Yes.
[1:03:14] A small batch.
[1:03:15] Uh, locally.
[1:03:17] Locally crafted knowledge.
[1:03:19] Yeah, exactly.
[1:03:20] So, so I think that my view is that not only is there a field that is like
[1:03:27] ML plus medicine that is medicine, I think that is the thing that
[1:03:34] medicine will be as a science now.
[1:03:37] It'll incorporate a bunch of other stuff.
[1:03:38] Like you need a bunch of causal inference stuff that's not.
[1:03:41] Really part of ml.
[1:03:43] You need behavioral science, you need economic, like
[1:03:46] there, there's a like a lot.
[1:03:47] But at its core, I think just given the data that we have, it's high dimensional.
[1:03:53] It's like it's everywhere.
[1:03:54] It's like super complicated and given the types of problems that medicine
[1:03:58] solves, which are, you know, really like often around prediction and
[1:04:01] description and accounting for variance.
[1:04:04] I think this is medicine.
[1:04:06] I think our common friend, Zach Johanne, who's the chair of the department at
[1:04:10] Harvard, has this phrase that I think we've all heard a thousand times.
[1:04:14] That just becomes like a chant at some point.
[1:04:16] Or a mantra, which is medicine is an information processing discipline.
[1:04:20] And I think that at its core, I think that aligns perfectly with what you're saying.
[1:04:24] That one, what medicine should be doing is processing information and it's
[1:04:28] kind of like this discipline of doing that and how well are we doing that
[1:04:31] is, is actually the the key question.
[1:04:34] Yeah, I love that.
[1:04:34] Yeah.
[1:04:36] All right, ed, we have one last question.
[1:04:38] It's a pair of questions inviting you to imagine two different scenarios here.
[1:04:43] The first is, what is your best case scenario for AI in healthcare?
[1:04:46] And then of course opposing that.
[1:04:48] What is your worst case or what are you most worried about for AI in healthcare?
[1:04:54] I'll start maybe with the worst case 'cause it's, it's
[1:04:57] easier to imagine because it's closer, unfortunately, to today.
[1:05:01] I think the worst case is it's not just that we get biased tools and,
[1:05:09] and deeply flawed tools of the kinds that we've talked about, that I've
[1:05:12] studied, that I'm really worried about.
[1:05:15] It's that all those tools are used for is this local optimization of a system.
[1:05:25] That sucks and that isn't proactive.
[1:05:28] That's very oriented towards billing and coding.
[1:05:32] There's a very unappealing path where we just get a hyper optimized
[1:05:39] version of our current shitty system.
[1:05:42] Can we count the beans faster?
[1:05:44] Yeah.
[1:05:44] Yeah, and, and I think that there's a certain.
[1:05:48] Lack of ambition in how people are applying AI today.
[1:05:54] That's really, I think, constrained by the imagination that people have
[1:05:59] and the kinds of problems that are front and center for decision makers
[1:06:02] today, which are unfortunately, you know, and, and you can't blame health
[1:06:07] system executives for being really worried about their profit margins.
[1:06:11] You know, given that inflation is really hard on hospitals that have fixed
[1:06:15] contracts, um, and, and rising costs.
[1:06:18] So it is just like, I, I get it, but it's not good.
[1:06:21] So I think that the optimistic version of that is that, man, there are a lot
[1:06:26] of great problems to solve with ai.
[1:06:28] And I think there are two types of problems, and I think I've
[1:06:31] worked on the former, but I'm starting to work on the latter.
[1:06:35] The former is just, there are just these first order misallocations
[1:06:39] of resources that are tragic.
[1:06:41] Like I, I am working on a project on the theme of getting data.
[1:06:46] Uh, this is with a, in collaboration with a regional health system in
[1:06:49] Sweden, because in Sweden you can get all of the electrocardiograms that were
[1:06:53] ever done in that region, uh, and you can link them to death certificates.
[1:06:57] And this is with a, a cardiologist in Sweden called Marcus Linman and
[1:07:01] my other co-author, Sendal Mohan.
[1:07:03] And we're studying sudden cardiac death.
[1:07:05] So sudden cardiac death is exactly what it sounds like.
[1:07:07] People just drop dead.
[1:07:09] It's really hard to know why even expost after they've dropped dead,
[1:07:13] because it's really hard to study people after they've dropped dead.
[1:07:17] And what makes it particularly tragic is that for at least some fraction
[1:07:19] of those deaths, we have the cure.
[1:07:22] We have defibrillators that we could put in people to terminate the fatal
[1:07:27] arrhythmias that cause these sudden cardiac deaths, but we just don't know
[1:07:31] who to put those defibrillators in.
[1:07:33] So one part of that project is just asking the very practical question
[1:07:36] of can we identify people who would benefit from defibrillators before
[1:07:40] they die rather than after they die?
[1:07:43] Super important, like very practical question around, we have
[1:07:47] this resource, we're currently putting it into the wrong people.
[1:07:50] 75% of defibrillators never terminate a fatal arrhythmia.
[1:07:54] It's crazy.
[1:07:56] Um, and on the other hand, lots of people are dying of arrhythmias that could
[1:08:00] be prevented by these defibrillators.
[1:08:01] So crazy Misallocation first order problem, hundreds of thousands
[1:08:05] of deaths in the US every year.
[1:08:07] So that's one kind of problem that I hope more people will work on.
[1:08:10] And to go back to Nightingale, the data sets that we put on Nightingale are.
[1:08:14] Constructed around exactly those kinds of problems that are interesting
[1:08:18] and important and, and critical.
[1:08:20] So I hope more people will go look at those data sets and
[1:08:22] start solving those problems.
[1:08:24] I think there's a second type of problem where AI can really help,
[1:08:28] which is not just the practical parts of like allocating resources in the
[1:08:32] healthcare system, but actually making scientific discoveries in health.
[1:08:37] And so we're applying our machine learning algorithms to E C G waveforms.
[1:08:43] So, you know, maybe the algorithm like sees a little squiggle, but.
[1:08:47] Unlike, you know, when you're doing this stuff in ImageNet, it's
[1:08:50] like, okay, yes, this is a Frisbee.
[1:08:52] That's why it's saying okay.
[1:08:54] But in ECGs, we actually know a lot about how the heart produces the E C G.
[1:08:59] We know what part of the heart leads to what part of the waveform.
[1:09:04] We have simulation models of the heart that we can get to produce waveforms.
[1:09:09] So I think tying that pipeline together of biological understandings of the
[1:09:14] heart and how the heart generates data to things that we're learning about the
[1:09:18] data via machine learning from patient outcomes, I think that's super promising.
[1:09:23] And I think that's the kind of stuff that I'm really excited about working
[1:09:25] on, um, over the next few years.
[1:09:27] So discovery of novel pathophysiology, um, in the heart from e c g data.
[1:09:33] Yeah, exactly.
[1:09:34] Yeah.
[1:09:34] Perfectly.
[1:09:34] But yeah, with, with a view.
[1:09:36] And then once you have that, Pipeline connected up.
[1:09:39] You can use it to discover drugs, you can tie it to animal models.
[1:09:43] You can, like, there's a lot of things you can do once you get the data.
[1:09:46] Talking to the biology.
[1:09:48] I have one wild card question to ask you at the end, just 'cause it
[1:09:51] ties in nicely with what you said.
[1:09:52] So, uh, you wrote a perspective in N E J M a few years back called Lost in Thought.
[1:09:58] And one of the points that you make in this perspective is that plausibly
[1:10:02] medicine already exceeds the capacity of the human mind and that we need
[1:10:07] AI as kind of like an integration tool to synthesize all this data.
[1:10:10] So can you sort of square that with what you just said?
[1:10:13] Can you, new pathophysiology.
[1:10:15] Understanding the goal, a goal relative to sort of the limitations of the human mind.
[1:10:23] I think, you know, you first have to give the human mind a lot of credit.
[1:10:27] Like we've figured out a lot of stuff about a very complicated system, but we
[1:10:32] really have no business figuring out.
[1:10:34] It's very striking, like when you take your car in.
[1:10:37] To the mechanic and you're like, I don't know.
[1:10:40] It's making this sound and the mechanic's like, yeah, I, I don't know.
[1:10:45] It's like we built the car.
[1:10:46] It is like we, we can't figure out the car.
[1:10:48] I mean, the body is pretty complicated.
[1:10:52] And I think that was the other thing that bothered me about med school is like, we
[1:10:55] have these drawings of like the cell and the nucleus and it's like, oh yeah, here
[1:10:59] comes the d n A, here's the ribosome.
[1:11:01] This is happening on the like millisecond scale.
[1:11:04] Things are bumping into each other all the time.
[1:11:06] We have no idea what's going on at, at a, at a macro level, and yet we still
[1:11:11] manage to figure out a bunch of stuff.
[1:11:13] It's amazing that we figured anything out that said we could use some help because,
[1:11:18] you know, If you think about something like a decision rule, you know, we use
[1:11:23] these decision rules in the hospital.
[1:11:24] It's like, okay, if your E K G does this and you're over this
[1:11:29] many years old and you're on this medication, then your score is this.
[1:11:32] But I don't think anyone actually believes that heart attack is a phenomenon
[1:11:37] that can be adequately described with like five integer weighted variables.
[1:11:42] It's just, you know, there are useful constructions that help,
[1:11:46] but things are complicated.
[1:11:48] We, we have this paper that looks at actually physicians in the emergency
[1:11:51] department diagnosing heart attack, and that finds that if you constrain
[1:11:56] a machine learning algorithm to a very small number of variables, it
[1:12:01] actually looks a lot like the doctor.
[1:12:03] So if we put ourselves on even ground, And are restricted to
[1:12:07] a small number of variables.
[1:12:08] We actually are doing pretty well.
[1:12:10] It's not optimal, but it's like pretty close.
[1:12:12] The problem with us is that we can't go up to like 10,000 variables.
[1:12:17] We are low dimensional creatures.
[1:12:18] Yeah.
[1:12:19] And medicine is a high dimensional field.
[1:12:22] And so I think being able to capture that high dimensionality while also
[1:12:28] getting it in dialogue with our ability to think about models and
[1:12:33] counterfactuals and reason through things like that's, I think the thing that's
[1:12:38] really gonna drive science forward.
[1:12:40] I think over the next few decades there's, there's a wonderful paper that
[1:12:43] you probably know already, but it's um, by my friend and colleague, Reina
[1:12:48] Barlay at m i t, where she and colleagues actually train an algorithm to discover
[1:12:54] new antibiotics in classes that are Yeah.
[1:12:57] Halon.
[1:12:57] That, that were completely unknown to people whose living
[1:13:01] is to discover antibiotics.
[1:13:03] And it all just came out of this amazing.
[1:13:05] Exercise that tied data discovery to new drug libraries, to an
[1:13:10] experimental model where you can see whether things are killing e coli.
[1:13:14] So I think that paper for me was really like opening the door to
[1:13:18] this new way of doing science that integrates human reasoning and data
[1:13:22] collection and hypothesis generation with data-driven approaches from ml.
[1:13:28] So yeah, I think that's a great note, uh, an optimistic one to end on as well.
[1:13:32] So I want to just thank you so much for being on AI Grand Rounds.
[1:13:35] Uh, this was a really, really fascinating conversation.
[1:13:39] This was so much fun.
[1:13:40] It's always great to hang out with both of you.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.