# OpenAI Reports New AI Safety Incidents; AI CEOs Weigh In on AI Debate

An episode of [Bloomberg Tech](https://invisiblenetwork.ai/podcast/1611767434), 17 Sep 2026, 43 min.

Audio: https://podtrac.com/pts/redirect.mp3/traffic.omny.fm/d/clips/e73c998e-6e60-432f-8610-ae210140c5b1/41764a4f-fc64-4e11-89ba-ae7c0030ab5e/b0a9874b-2305-4a24-8e3a-b4c801372d56/audio.mp3?utm_source=Podcast&in_playlist=9caafc41-289c-4115-995d-ae7c0030ab75

Bloomberg’s Ed Ludlow breaks down OpenAI's new plan to track when things go wrong after revealing new cases of its AI models going off-script. Plus, Databricks CEO Ali Ghodsi and machine learning pioneer Andrew Ng weigh in on the global AI safety debate; meanwhile in Silicon Valley, the immediate fear is missing out on the money. See omnystudio.com/listener for privacy information.

## Transcript

[0:02] Speaker 1: Bloomberg Audio Studios.
[0:04] Speaker 2: Podcasts.
[0:05] Speaker 3: Radio.
[0:06] Speaker 2: News.
[0:13] Speaker 4: Bloomberg Tech is live from the heart of Silicon Valley
[0:17] Speaker 4: with Ed Ludlow in San Francisco.
[0:23] Speaker 5: This is Bloomberg Tech.
[0:24] Speaker 6: Coming up, OpenAI reveals new cases of its AI models
[0:27] Speaker 6: misbehaving and a new plan to track when things go wrong. Plus,
[0:31] Speaker 6: Databricks CEO Ali Ghodzi and machine learning pioneer Andrew Ng
[0:35] Speaker 6: weigh in on the global AI safety debate. And while
[0:38] Speaker 6: AI may pose an existential risk in Silicon Valley, the
[0:42] Speaker 6: immediate fear is missing.
[0:44] Speaker 5: Out on the money.
[0:45] Speaker 6: That story later this hour. AI warnings take the global
[0:50] Speaker 6: stage as King Charles III sounds the alarm.
[0:54] Speaker 7: There seems urgency in adequately considering the existential dangers of
[1:01] Speaker 7: such technologies falling into the wrong hands and being used
[1:06] Speaker 7: in potentially catastrophic ways. Surely then we need sufficient means
[1:13] Speaker 7: of control before it is all too late.
[1:17] Speaker 6: That was King Charles pressing some of AI's biggest names
[1:20] Speaker 6: for reassurance they can keep AI under control. NVIDIA's Jensen Wong,
[1:24] Speaker 6: OpenAI's Sarah Fryer, and DeepMind's Demis Hassabis were among those
[1:28] Speaker 6: gathered in Scotland for the event. I want to get
[1:31] Speaker 6: to Bloomberg Tech Editor Shona Ghosh over in London with
[1:34] Speaker 6: the latest. Let's start with why King Charles pulled this event.
[1:38] Speaker 5: Together, Shona.
[1:39] Speaker 4: Yes.
[1:39] Speaker 8: I mean, it's not an obvious choice, is it, to
[1:41] Speaker 8: convene an AI gathering? But, you know, still a monarch,
[1:44] Speaker 8: still has the sway. Ultimately, it's against the backdrop of,
[1:49] Speaker 8: you know, Anthropic CEO Dario Amadei's blog post over the weekend.
[1:54] Speaker 8: saying that he thinks there should be a pause, a
[1:56] Speaker 8: slowdown in frontier AI development. That's been accompanied separately by
[2:01] Speaker 8: admittedly lower level, but AI researchers quitting some of the
[2:05] Speaker 8: top AI companies, including Anthropic, also DeepMind, effectively saying that
[2:10] Speaker 8: AI could be lethal for humans. So There's a sense
[2:13] Speaker 8: that fear around AI has been a debate within Silicon Valley,
[2:18] Speaker 8: but actually now we're seeing those fears spill out beyond
[2:21] Speaker 8: Silicon Valley. MPs, lawmakers in the UK and globally are
[2:25] Speaker 8: quite concerned. The general public are also quite concerned about
[2:28] Speaker 8: what happens if AI isn't adequately controlled. And again, that's
[2:32] Speaker 8: all been triggered in turn by events over the summer
[2:35] Speaker 8: where we saw AI agents from OpenAI, you know, the
[2:39] Speaker 8: phrase is going rogue. I wouldn't say that's quite right. But,
[2:42] Speaker 8: you know, escaping guardrails and attacking other organizations in ways that,
[2:47] Speaker 8: you know, OpenAI wasn't necessarily expecting.
[2:50] Speaker 6: Shona, remind us which leaders of industry were in the room,
[2:53] Speaker 6: so to speak, and what King Charles's message to them was,
[2:57] Speaker 6: maybe even his instruction to them was.
[3:01] Speaker 8: So gathered some big names. We had NVIDIA CEO Jensen Wong,
[3:07] Speaker 8: the former CEO and co-founder of DeepMind, Demis Hassabis. Also,
[3:12] Speaker 8: Sarah Fryer, chief financial officer of OpenAI. Anthropic did have
[3:16] Speaker 8: a representative in the room, but it was their global
[3:18] Speaker 8: policy chief. It wasn't any of the co-founders.
[3:20] Speaker 5: It wasn't Dario.
[3:21] Speaker 8: So kind of missing a big name there. Also, no
[3:23] Speaker 8: one from Meta or X, interestingly. So, you know, good representation,
[3:28] Speaker 8: but not everyone. And the monarch prepared a speech that
[3:32] Speaker 8: was sort of released in advance. And essentially, you know,
[3:35] Speaker 8: he told these leaders that, Something he said before, actually,
[3:38] Speaker 8: which is, you know, if you're going to develop this tech,
[3:40] Speaker 8: can you make sure that it's good for humanity, good
[3:44] Speaker 8: for the natural world, which is, you know, environmentalism is
[3:47] Speaker 8: something King Charles has particularly been interested in and has
[3:50] Speaker 8: been for a long time. So he was really pushing
[3:51] Speaker 8: for the technology to benefit humanity and to just pause
[3:54] Speaker 8: and think about those consequences.
[3:57] Speaker 6: Bloomberg Show Negotiation London, thank you very much. Another big story,
[4:00] Speaker 6: OpenAI is revealing previously unreported cases of its AI models
[4:04] Speaker 6: going off-script, fabricating data, bypassing restrictions, sharing files they were
[4:10] Speaker 6: told to keep private. Now the company is setting up
[4:13] Speaker 6: a new system to track and disclose those incidents. Bloomberg's
[4:16] Speaker 6: Shireen Ghaffari, who leads our coverage with Frontier Labs, is
[4:19] Speaker 6: with us. Let's start with the plan that OpenAI has
[4:22] Speaker 6: presented on what it's going to do if it finds
[4:25] Speaker 6: models misbehaving.
[4:26] Speaker 5: What have they said? That's right.
[4:29] Speaker 9: So they have said that they are now going to
[4:31] Speaker 9: essentially have a triage-like system for employees to disclose incidents
[4:37] Speaker 9: where they think that the AI models are essentially misbehaving,
[4:41] Speaker 9: are eliciting some kind of concerning behavior that goes against
[4:45] Speaker 9: what they intend the AI to do.
[4:48] Speaker 6: There were some interesting case studies. OpenAI also made it
[4:51] Speaker 6: clear that in the six they disclosed, there was no
[4:54] Speaker 6: hacking of third parties. But some of them are worrying, right?
[4:58] Speaker 6: Take us through what OpenAI disclosed this morning.
[5:01] Speaker 8: Yeah.
[5:01] Speaker 9: So even though when we asked, OpenAI said that this
[5:04] Speaker 9: did not involve these incidents, six that they have disclosed
[5:07] Speaker 9: did not involve reaching third parties. There are still some
[5:11] Speaker 9: concerning behaviors that we're seeing here. For example, AI models
[5:15] Speaker 9: trying to share files that should have been private with
[5:18] Speaker 9: each other, communicating with each other in ways they're.
[5:21] Speaker 1: Not supposed to.
[5:23] Speaker 9: Telling AI models essentially, learning how to disregard or instructing
[5:28] Speaker 9: others to disregard their instructions.
[5:30] Speaker 1: On how to behave.
[5:32] Speaker 9: So these are all incidents where you can see the
[5:35] Speaker 9: potential for that to evolve into something even more nefarious,
[5:38] Speaker 9: especially if those AIs were successful and sort of going
[5:42] Speaker 9: against what the creators of the AI expect these AIs
[5:46] Speaker 9: to do.
[5:47] Speaker 6: Your report is one of the most read on Bloomberg
[5:49] Speaker 6: this morning. That is not surprising. And lower down in
[5:52] Speaker 6: the report and in OpenAI's blog post, there is an
[5:56] Speaker 6: admittance that the state of play in industry from safety
[6:00] Speaker 6: and alignment is not where it should be.
[6:02] Speaker 5: What did they say?
[6:04] Speaker 9: That's right. Essentially, nobody has solved this problem of so-called
[6:08] Speaker 9: AI alignment, which just means, is AI actually working in
[6:13] Speaker 9: accordance with the intended outcome or how humans would want
[6:19] Speaker 9: AI to behave? And the fact that even the brightest
[6:22] Speaker 9: minds in the world, that even the top AI companies
[6:25] Speaker 9: have not figured this out, is concerning by these companies'
[6:30] Speaker 9: own admission. And so, You know, they're saying that they
[6:33] Speaker 9: hope to disclose more of these incidents publicly, but there
[6:36] Speaker 9: are still a lot of unanswered questions. And there may
[6:39] Speaker 9: be many more of these incidents that we just don't
[6:41] Speaker 9: know about yet.
[6:43] Speaker 6: Bloomberg Shireen Ghaffari, thank you very much. We'll have more
[6:45] Speaker 6: on the growing AI warnings later this hour. We're going
[6:48] Speaker 6: to speak to Ali Goldzi, the Databricks CEO. And then
[6:50] Speaker 6: even later in the hour, Andrew Ng, founder of deeplearning.ai,
[6:55] Speaker 6: a long-term researcher.
[6:57] Speaker 5: In the field of artificial intelligence.
[7:00] Speaker 6: And even as those safety concerns grow, the money keeps
[7:03] Speaker 6: flowing into open AI. Apollo's in talks with SoftBank to
[7:07] Speaker 6: boost a loan to as much as $ 9 billion, helping
[7:11] Speaker 6: finance the Japanese firm's massive bet on the AI company.
[7:14] Speaker 5: That's according to sources.
[7:15] Speaker 6: SoftBank's committed nearly $ 65 billion to open AI and has
[7:19] Speaker 6: increasingly turned to borrowing to fund that investment. And that
[7:24] Speaker 6: massive flow of money into AI is fueling demand across
[7:27] Speaker 6: the industry. Nvidia CEO Jensen Wang says the opportunity is
[7:30] Speaker 6: expanding across countries and economies.
[7:34] Speaker 10: I expect Nvidia to sell twice as many chips this
[7:37] Speaker 10: next year as we do this year. And the reason
[7:40] Speaker 10: for that is because AI has so much contribution to
[7:45] Speaker 10: the benefits of different industries, different economies. And you could
[7:49] Speaker 10: see that almost every single country that we're in, people
[7:52] Speaker 10: want to invest in AI.
[7:55] Speaker 5: Joining us now, T.
[7:56] Speaker 6: Rowe Price, portfolio manager, Tony Wong. NVIDIA is the top
[7:59] Speaker 6: holding with Tony. I wanted to play that soundbite because
[8:03] Speaker 6: to me, that sounds like a disclosure of unit growth,
[8:05] Speaker 6: which I haven't heard from NVIDIA. Often, if at all,
[8:10] Speaker 6: twice as many chips next year as this year. He
[8:11] Speaker 6: could be talking GPUs, the Grok LPU, CPUs, but just
[8:15] Speaker 6: react to that, Tony.
[8:18] Speaker 3: Yeah, I mean, for a company of that size and
[8:21] Speaker 3: scale to be growing like that, I mean, it's pretty
[8:23] Speaker 3: much unprecedented. And I think that you have to look
[8:25] Speaker 3: at what's really going on in demand. I think we
[8:28] Speaker 3: are seeing inflection and agentic adoption in enterprise. And you
[8:33] Speaker 3: see releases of personal agentic use cases with views and instincts.
[8:40] Speaker 3: And so I think that this is what's going on
[8:42] Speaker 3: in terms of We're going from AI answering questions to
[8:45] Speaker 3: actually doing actions and completing tasks and producing outcomes. And
[8:50] Speaker 3: so I think that there's a lot going on here.
[8:52] Speaker 3: And it makes sense that NVIDIA, as one of the
[8:55] Speaker 3: biggest platforms for AI computing, is going to be a
[8:58] Speaker 3: big part of driving that.
[9:00] Speaker 5: Tony, I'm going to take you back to the weekend.
[9:03] Speaker 6: Industry supporting each other in saying we need to pace
[9:06] Speaker 6: at the frontier. Now bring us back to public markets today.
[9:11] Speaker 6: How did that cause you and your industry to kind
[9:13] Speaker 6: of remodel demand going forward? Is there any change at
[9:16] Speaker 6: all on that idea that there would be a slowing
[9:19] Speaker 6: or a pacing at the frontier?
[9:21] Speaker 3: Yeah, look, I think that safety is definitely a very
[9:25] Speaker 3: important topic. And I think it's good that the industry
[9:27] Speaker 3: is super focused on that.
[9:29] Speaker 5: And there's a lot of tension around it.
[9:30] Speaker 3: And whenever you're developing a new technology, you want to
[9:33] Speaker 3: make sure there's guardrails in place. And this is obviously
[9:37] Speaker 3: the most critical technology in the world right now.
[9:39] Speaker 5: So to me, I think it's good. And I think
[9:42] Speaker 5: at the same.
[9:42] Speaker 3: Time, it's also not an aspect of like, we're going
[9:44] Speaker 3: to stop. developing frontier models. If anything, that's continued to
[9:49] Speaker 3: be very top-minded. And the way to do this is
[9:53] Speaker 3: not to pause the AI kind of development, but to
[9:58] Speaker 3: make sure that we're all paying attention to what it
[10:01] Speaker 3: is and like to make sure that we put investment
[10:03] Speaker 3: behind it. So to me, I think there's areas of
[10:05] Speaker 3: cybersecurity that are super important that are becoming more important.
[10:08] Speaker 3: And then, you know, overall, I think that it just
[10:11] Speaker 3: underscores how important this is.
[10:14] Speaker 5: In addition, keeping the frontier, you know, race going. I
[10:19] Speaker 5: study your portfolio.
[10:20] Speaker 6: So, you know, it is NVIDIA and Broadcom and then
[10:23] Speaker 6: Apple and the hyperscalers up there, right? You pose the question,
[10:27] Speaker 6: who owns the customer when the agent owns the interface?
[10:30] Speaker 6: I don't see that much of a software play in
[10:33] Speaker 6: the holdings, at least the top ones, Tony.
[10:37] Speaker 3: Yeah, well, I think that, you know, I think that's
[10:39] Speaker 3: a great thing that the market is thinking about is that,
[10:42] Speaker 3: you know, over the last 20 years, the Internet has
[10:45] Speaker 3: um companies have essentially you know become like the interfaces
[10:49] Speaker 3: between customers and then companies that deliver product and so
[10:53] Speaker 3: i think the agents kind of change things in terms
[10:55] Speaker 3: of like you know for a consumer like instead of
[10:57] Speaker 3: going logging on clicking on a bunch of things going
[11:00] Speaker 3: to various websites comparing like you can have a personal
[11:03] Speaker 3: agent that kind of does that for you and brings
[11:05] Speaker 3: you the results right so i think that's going to
[11:07] Speaker 3: be an interesting paradigm shift in terms of you know,
[11:10] Speaker 3: it'll be more important for software companies and, you know,
[11:14] Speaker 3: kind of OTAs to like really embrace this change. And
[11:18] Speaker 3: I think that it won't be all the same, you know,
[11:20] Speaker 3: companies and software companies, especially that have data gravity, that
[11:24] Speaker 3: have the workflow. You know, the orchestration, they understand the customers,
[11:28] Speaker 3: they know the security, they know the permissioning. They're going
[11:31] Speaker 3: to be important in terms of, you know, these models
[11:34] Speaker 3: are kind of like the brain. And then you also
[11:36] Speaker 3: have to develop the body and the nervous system to
[11:38] Speaker 3: make sure you can carry out that action. So I
[11:40] Speaker 3: think it'll be a really interesting area to watch because
[11:44] Speaker 3: a lot of those companies have been gone from like
[11:47] Speaker 3: kind of AI loser bucket to the AI winner bucket,
[11:50] Speaker 3: you know. So you see tremendous moves and that's exciting
[11:53] Speaker 3: to see.
[11:54] Speaker 5: Tony Wong of T. Rowe Price, really great to have
[11:56] Speaker 5: you back on the show. Thank you.
[11:58] Speaker 6: Some of tech's biggest names are heading to the White House,
[12:00] Speaker 6: by the way. Apple executive chairman Tim Cook, OpenAI CEO
[12:04] Speaker 6: Sam Altman, are set to attend President Trump's state dinner
[12:07] Speaker 6: for Chinese President Xi Jinping, which is next week. NVIDIA
[12:11] Speaker 6: CEO Jensen Wang is also expected to attend. ZipLine could
[12:17] Speaker 6: be getting a major valuation boost. The drone delivery startups
[12:20] Speaker 6: in talks to raise about a billion dollars at a
[12:22] Speaker 6: roughly $ 20 billion valuation, nearly triple what it was worth
[12:26] Speaker 6: earlier this year, all according to sources. Bloomberg's Natasha Mascarinas
[12:30] Speaker 6: broke the story with me and joins us here on set.
[12:32] Speaker 5: Let's start with the round. What do we know? What
[12:34] Speaker 5: do we not? Don't know. Yeah, absolutely.
[12:36] Speaker 11: So earlier this year, ZipLine raised at around a $ 7. 6
[12:38] Speaker 11: billion valuation and then came back in the news last
[12:42] Speaker 11: month to announce a new Uber partnership that was set
[12:44] Speaker 11: to expand drone delivery's in the United States. And so
[12:47] Speaker 11: now we have news on how investors are perceiving that update,
[12:51] Speaker 11: which is with more capital.
[12:53] Speaker 1: They're in talks to.
[12:54] Speaker 11: Raise around $ 1 billion, like you said, at around a
[12:56] Speaker 11: $ 20 billion valuation. Still pretty much in flux in terms
[13:00] Speaker 11: of who all is participating, but we do have some
[13:02] Speaker 11: names like Paradigm and Tegra Global involved in the round,
[13:06] Speaker 11: both existing investors.
[13:08] Speaker 6: It sounds early. stage-ish in this particular round to me.
[13:11] Speaker 6: I mean, the origin of Zipline is a company that,
[13:14] Speaker 6: particularly in the continent of Africa, was about finding a
[13:17] Speaker 6: solution to get blood medication to remote areas where there's
[13:20] Speaker 6: no hospital. And then it evolved to delivery. It has
[13:24] Speaker 6: a relationship with Walmart. What is Zipline? Tell us about
[13:27] Speaker 6: the company a little bit.
[13:28] Speaker 5: Yeah, it's so interesting.
[13:29] Speaker 11: This is not a company in the AI zeitgeist like
[13:31] Speaker 11: you and me are usually talking about here, but this
[13:34] Speaker 11: is a company founded in 2014, spent a decade building
[13:37] Speaker 11: a logistics network, and I think of them as an
[13:40] Speaker 11: example of this whole vision of vertical technology. build the hardware,
[13:45] Speaker 11: like the drones themselves, and then they also build the software,
[13:48] Speaker 11: the logistics network, et cetera, et cetera, which is why
[13:50] Speaker 11: they can go from delivering blood to delivering, you know,
[13:53] Speaker 11: Jimmy Johns.
[13:53] Speaker 6: Right, we're showing the video now of the kind of
[13:56] Speaker 6: illustrative examples of their sort of home delivery. I've also
[13:59] Speaker 6: been out to Benville, Arkansas, and kind of seen it
[14:02] Speaker 6: firsthand with their prior generation.
[14:04] Speaker 5: Paradigm, I asked you.
[14:05] Speaker 6: About this last night on the phone, like Paradigm, not
[14:07] Speaker 6: a firm that I'm as familiar with. We understand in
[14:10] Speaker 6: our reporting they're leading this round.
[14:12] Speaker 11: Yeah, so I've been following them for quite a bit.
[14:13] Speaker 11: I mean, they are founded by a former Sequoia investor
[14:17] Speaker 11: and have actually made their name first and foremost in crypto.
[14:21] Speaker 11: But earlier this year, they announced that they were raising
[14:24] Speaker 11: their fourth fund, I believe a little bit over $ 1 billion,
[14:27] Speaker 11: and expanding into frontier technology, which includes AI, includes robotics.
[14:33] Speaker 11: They're an existing investor in Zipline, but I do think
[14:35] Speaker 11: this is an example of a once crypto firm making
[14:39] Speaker 11: like one of their biggest bets this year, I would assume,
[14:42] Speaker 11: on a company outside of crypto.
[14:43] Speaker 5: Bloomberg's Natasha Mascarenas. Thank you very much indeed.
[14:46] Speaker 6: OK, coming up, another layer of the tech stack weighs
[14:50] Speaker 6: in on the AI pacing debate. Databricks CEO Ali Goldzi
[14:54] Speaker 6: joins us next. This is Bloomberg Tech.
[15:13] Speaker 10: As we move from labs to product development, the companies
[15:18] Speaker 10: have to become much more rigorous in testing and securing
[15:23] Speaker 10: and making sure that the products are ready for use
[15:26] Speaker 10: before it's released. and in doing so if it's not
[15:31] Speaker 10: ready just hold it back you should go as fast
[15:34] Speaker 10: as you can but no faster than.
[15:36] Speaker 6: That NVIDIA's Jensen Wang, again, speaking ahead of that meeting
[15:39] Speaker 6: with King Charles III. OpenAI has raised the ante on
[15:43] Speaker 6: safety worries with its latest admission that its models misbehaved
[15:46] Speaker 6: and saying in its blog post that the industry has
[15:49] Speaker 6: not solved alignment and monitoring to a sufficient degree to
[15:53] Speaker 6: continue responsibly scaling at maximum speed for much longer. I
[15:57] Speaker 6: want to get the perspective from another layer in the
[15:59] Speaker 6: AI stack, and that is from Ali Godzi, the CEO
[16:02] Speaker 6: of Databricks. Pacing at the frontier and where we're at
[16:05] Speaker 6: in alignment and safety weigh.
[16:08] Speaker 1: In, Ali.
[16:10] Speaker 12: Yeah, I think first and foremost, as leaders, we have
[16:14] Speaker 12: a responsibility to not freak people out. So I think
[16:17] Speaker 12: it's really important that we tell people the existential risk
[16:21] Speaker 12: to humanity is close to zero. So no one needs
[16:24] Speaker 12: to lose sleep over that. I think this is important
[16:25] Speaker 12: because I think a lot of people, you know, they
[16:28] Speaker 12: might get freaked out about these things and it might
[16:29] Speaker 12: put them in a bad state. And it actually can
[16:31] Speaker 12: have serious consequences on people's health, mental health out there.
[16:35] Speaker 5: So the existential, like.
[16:37] Speaker 12: You know, hey, you know, the humanity is, you know,
[16:40] Speaker 12: AI takes over. That's that probably is close to zero.
[16:44] Speaker 12: Where there is risk is in cyber, because AI is
[16:48] Speaker 12: just very good at breaking into things. So it's very
[16:50] Speaker 12: similar to the very early days of the Internet.
[16:53] Speaker 5: When we had.
[16:54] Speaker 12: These worms that were going around and we had viruses
[16:56] Speaker 12: that were spreading because we just hadn't connected the whole
[17:00] Speaker 12: world together through the internet. Once we did that, suddenly
[17:03] Speaker 12: you could attack any target anywhere. And it's a similar
[17:06] Speaker 12: situation now where the AI models are just great at
[17:08] Speaker 12: breaking in and now they have all these targets to
[17:11] Speaker 12: go through. So we do actually have to have really,
[17:13] Speaker 12: really huge investments in cyber and protect our software systems
[17:19] Speaker 12: that exist all around the planet.
[17:20] Speaker 13: This is real.
[17:21] Speaker 12: And this we have to do. And actually, we are
[17:23] Speaker 12: actually in that space. We have a product called Lakewatch
[17:25] Speaker 12: that actually helps you do that.
[17:26] Speaker 5: And it uses agents.
[17:28] Speaker 12: It uses AI to actually do that. That is real.
[17:31] Speaker 12: And it will have damages and costs to it. But
[17:33] Speaker 12: that's not an existential risk.
[17:36] Speaker 5: The risk.
[17:37] Speaker 6: Can I take you back to kind of the early
[17:38] Speaker 6: days of the pandemic? At the time, you were talking
[17:42] Speaker 6: about being quite paranoid, right? In general, not about AI
[17:46] Speaker 6: at that time. So you took your teams and have done.
[17:51] Speaker 6: into some sky is falling scenarios. You basically want your
[17:54] Speaker 6: people to prepare for the worst possible case scenarios that
[17:57] Speaker 6: could happen in the world. Are you still doing that?
[18:00] Speaker 6: And how is that apply in this AI safety context?
[18:04] Speaker 12: Yeah, I've always been one of these, you know, really,
[18:06] Speaker 12: really paranoid CEOs. We've been doing AI, you know, since
[18:08] Speaker 12: the first days, 2013. Our first use case was.
[18:11] Speaker 5: AI use case.
[18:12] Speaker 12: So yeah, we do those every year and we want
[18:14] Speaker 12: to make sure that we're protected. So as I said,
[18:16] Speaker 12: I think cyber is real. So that is actually part
[18:18] Speaker 12: of our exercises. You know, what happens if someone gets
[18:21] Speaker 12: access to the computer systems? Like we have very, very
[18:23] Speaker 12: sensitive data at Databricks. So we actually do a lot
[18:26] Speaker 12: of these simulations of skies falling also on the cyber side.
[18:30] Speaker 12: And we let basically attackers from outside try to break
[18:33] Speaker 12: in and get all the way into the most sacred
[18:37] Speaker 12: secrets of Databricks and keep hardening those. And actually, we've
[18:40] Speaker 12: had to ramp up those efforts, I would say, in
[18:42] Speaker 12: the last four years. 2018, 2019, the time it would
[18:47] Speaker 12: take for an attacker, from revealing a vulnerability and some
[18:52] Speaker 12: attacker weaponizing it, it means actually breaking into a site,
[18:56] Speaker 12: would be two years. This has now shrunk down to hours.
[19:00] Speaker 12: So the moment a vulnerability is out, in less than hours,
[19:04] Speaker 12: someone will be attacked with that. So this is real.
[19:08] Speaker 12: That's just not existential. So I just think we should
[19:10] Speaker 12: separate these things. People are mixing these things and making
[19:14] Speaker 12: people worried about AI because AI has a lot.
[19:18] Speaker 5: Of great use cases.
[19:18] Speaker 12: We see huge positive use cases in our customer base
[19:21] Speaker 12: where they're actually moving the needle. That actually means a
[19:24] Speaker 12: lot to people to be able to leverage that AI.
[19:28] Speaker 6: Ali, in June, when we spoke, you said, you know,
[19:30] Speaker 6: it's not a good year for an IPO because of
[19:32] Speaker 6: the pipeline of huge offerings, right? If you are the
[19:35] Speaker 6: CEO or CFO for a private company, like looking at
[19:37] Speaker 6: the market. But this past week, you know, simply, and
[19:41] Speaker 6: I know your investors won't thank you for the answer
[19:43] Speaker 6: to this, but Sam Altman said to Fortune at the weekend,
[19:47] Speaker 6: IPO next year because of AI safety concerns.
[19:50] Speaker 5: Right now. Does that factor into your calculus?
[19:53] Speaker 12: Yeah, I mean, look, there's just so many, so much
[19:56] Speaker 12: demand for just the software. So like getting bogged down
[20:00] Speaker 12: in preparing an IPO for us, it just wouldn't be a,
[20:03] Speaker 12: you know, it wouldn't be good use of our time.
[20:05] Speaker 12: We will go public, but it's just, you know, we
[20:07] Speaker 12: just have to, we want to rather focus on our
[20:09] Speaker 12: business right now. I mean, you had just before this,
[20:12] Speaker 12: you had Zipline on. Zipline is a big customer of Databricks.
[20:14] Speaker 12: They have amazing use cases. That's an AI use case,
[20:17] Speaker 12: where they use AI to automate all of the drones,
[20:21] Speaker 12: the medicine delivery. Another awesome use case is Crisis Text Line.
[20:25] Speaker 12: They actually use AI to detect self-harm among the youth.
[20:29] Speaker 12: And they can actually use these large language models to
[20:31] Speaker 12: detect that and help prevent that. There's so much demand
[20:34] Speaker 12: for this stuff. And then I mentioned the cyber use
[20:37] Speaker 12: cases that we have. We'd rather be focused on this.
[20:40] Speaker 12: If we were public right now, we would be spending
[20:41] Speaker 12: all our time trying to figure out How are we
[20:43] Speaker 12: going to react to this, you know, latest crisis? And
[20:46] Speaker 12: do people think that, you know, the world is going
[20:49] Speaker 12: under and, you know, what are we going to do
[20:50] Speaker 12: about this? And we'd have to worry about our stock price.
[20:52] Speaker 12: So I think in these big times of transition, you know,
[20:56] Speaker 12: it's better to do those transitions in private. In fact,
[20:58] Speaker 12: some public companies go private during times of transition to
[21:01] Speaker 12: do those transitions and then they go public again. So
[21:04] Speaker 12: we will be public. I just think the timing is
[21:06] Speaker 12: not good now.
[21:07] Speaker 6: Just to clarify quickly, we didn't have a zip line
[21:09] Speaker 6: on the show. We did have some reporting about their
[21:11] Speaker 6: latest private market round. Let's end on a positive, right?
[21:15] Speaker 6: In this environment where data is quite clearly key to
[21:17] Speaker 6: the harness on a model, the utility and capability of.
[21:21] Speaker 5: Model, where is Databricks growing? Where are you seeing some momentum?
[21:26] Speaker 12: Yeah, I mean, honestly, we're seeing acceleration across the board. Really,
[21:29] Speaker 12: we've tried to pin down where is the acceleration? Is
[21:31] Speaker 12: there a particular thing? And I think what's just happening
[21:34] Speaker 12: is that all these use cases that I mentioned, it's
[21:37] Speaker 12: almost as if every company on the planet doubled their
[21:40] Speaker 12: employee count because there's no agents working at all these companies.
[21:43] Speaker 12: And these agents want to do more things with data
[21:46] Speaker 12: and they get more insights from that data. And that
[21:49] Speaker 12: then increases the consumption that we're getting. Since our pricing
[21:53] Speaker 12: model is consumption-based, our revenue then goes up and it accelerates.
[21:58] Speaker 12: And in particular, we have one particular product called Genie,
[22:01] Speaker 12: which basically is your analyst. You can ask it anything
[22:03] Speaker 12: about your business.
[22:04] Speaker 5: How is this product doing?
[22:05] Speaker 10: Who's churning?
[22:06] Speaker 12: Where is my pipeline? And this product has accelerated the
[22:09] Speaker 12: revenue growth significantly.
[22:11] Speaker 5: So it's exciting times.
[22:14] Speaker 6: Databricks CEO Ali Gadzid, it's great to catch up. Thank
[22:16] Speaker 6: you for your time on Bloomberg Tech. I would just
[22:18] Speaker 6: point out this is what financial markets look like, particularly
[22:20] Speaker 6: the technology sector post-Fed Day, where the Fed indeed did
[22:24] Speaker 6: raise rates. You see some outperformance in technology. The Nasdaq
[22:28] Speaker 6: 100 up 1.6%. Outperformance in semiconductors. NVIDIA, a big contributor
[22:32] Speaker 6: to that. We played the soundbite earlier in the show,
[22:34] Speaker 6: but Jensen Wang, the CEO of NVIDIA, saying that they will.
[22:37] Speaker 5: sell twice.
[22:38] Speaker 6: As many chips next year as they did this year,
[22:41] Speaker 6: which to me seems like a material disclosure on unit growth.
[22:44] Speaker 6: US 10-year yield below 5%.
[22:47] Speaker 5: Pretty large move.
[22:48] Speaker 6: We've seen that across the bond market this morning. Let's
[22:51] Speaker 6: get back to our top story. AI is helping companies
[22:53] Speaker 6: build faster. But as competition at the frontier intensifies, should
[22:57] Speaker 6: there be limits on how much risk companies can take
[23:00] Speaker 6: to stay ahead? Karen McCormick, chief investment officer at Beringia,
[23:04] Speaker 6: argues the answer isn't slowing innovation, but establishing common guardrails
[23:08] Speaker 6: around testing, cybersecurity and incident reporting.
[23:11] Speaker 5: She joins us now. Explain that argument.
[23:15] Speaker 11: Sure.
[23:16] Speaker 13: And I just preface this by saying, so Beringia, we're
[23:19] Speaker 13: early growth investors. So our portfolio companies range in revenue
[23:23] Speaker 13: anywhere from one to 300 million.
[23:25] Speaker 1: So they'll be at the smaller end of the spectrum.
[23:27] Speaker 13: So we're not going to be investors in companies like
[23:29] Speaker 13: Anthropic or OpenAI, but our portfolio will be on the
[23:32] Speaker 13: receiving end of what they're publishing. So I guess our
[23:35] Speaker 13: perspective comes from those types of businesses. And I would
[23:39] Speaker 13: say the most important things to our companies are things
[23:42] Speaker 13: like what's the enterprise capability of what's being released? What's
[23:45] Speaker 13: the speed at which they're operating? What are the new
[23:47] Speaker 13: features that we're seeing? And really importantly, cost. Bundle all
[23:51] Speaker 13: of that with, is this product safe to use? Because
[23:54] Speaker 13: for companies like ours, we don't necessarily have.
[23:57] Speaker 1: large teams of.
[23:58] Speaker 13: Either security or cybersecurity or even IT. So we're really
[24:01] Speaker 13: reliant on what they're releasing for our portfolio companies who
[24:05] Speaker 13: have been net beneficiaries of all of it, but still concerned.
[24:09] Speaker 5: So that's why I think we should talk about regulatory capture.
[24:13] Speaker 9: Right.
[24:13] Speaker 6: Earlier this week, I was talking to David Sachs about
[24:16] Speaker 6: why there should or should not be a greater degree
[24:19] Speaker 6: of government intervention or regulation. And his viewpoint is right.
[24:21] Speaker 6: The frontier labs, they know what they're doing. They should
[24:24] Speaker 6: take on the responsibility of safety themselves. But that regular capture,
[24:29] Speaker 6: regulatory capture, how an idea for the whole industry serves
[24:33] Speaker 6: maybe just to how would that impact your portfolio?
[24:37] Speaker 1: Yeah, it's a good question.
[24:37] Speaker 13: It was a great interview, and I don't often agree
[24:39] Speaker 13: with David, but I did in this particular instance. It
[24:42] Speaker 13: does feel like we have, especially the two front runners,
[24:45] Speaker 13: Anthropic and OpenAI, who are on the one hand saying
[24:48] Speaker 13: we really need more safety guardrails, and on the other hand,
[24:51] Speaker 13: still continuing to produce at pace. So I do agree
[24:54] Speaker 13: with him that there needs to be some sort of self-regulation,
[24:57] Speaker 13: and our companies need to be able to trust that
[24:59] Speaker 13: the products that they're using are going to be safe.
[25:02] Speaker 13: Not the least because due diligence, when people are looking
[25:04] Speaker 13: at investing in our companies or acquiring them, they need
[25:07] Speaker 13: to have some kind of safety due diligence and guardrails
[25:11] Speaker 13: in place themselves. So, I think I overall agree with
[25:15] Speaker 13: the sentiment that there can be some self-policing in the industry. However,
[25:19] Speaker 13: one of the bigger risks is when things go wrong,
[25:22] Speaker 13: what we tend to find is that government gets more involved.
[25:25] Speaker 13: And government regulation for our portfolio companies often means more cost,
[25:29] Speaker 13: slower dispatches and just higher costs, less competition.
[25:33] Speaker 6: We're talking about early stage investing in AI in private markets, right?
[25:37] Speaker 5: Is there any.
[25:37] Speaker 6: Risk that the environment we're in results in your section
[25:42] Speaker 6: of the venture markets slowing down? Just stop writing checks.
[25:45] Speaker 5: For a while.
[25:47] Speaker 13: I mean, I would say we already are seeing some
[25:49] Speaker 13: of that kind of venture capitalists and private equities sitting
[25:52] Speaker 13: on our hands. I mean, it started with SaaSpocalypse and,
[25:55] Speaker 13: you know, are all these companies going to be decreasing
[25:58] Speaker 13: in value because AI can do all of the things
[25:59] Speaker 13: that they were doing?
[26:00] Speaker 1: We're finding that that didn't really happen.
[26:02] Speaker 13: There wasn't quite the SaaSpocalypse that we talked about, but
[26:04] Speaker 13: there's definitely a lot of delayed decisions waiting to see
[26:07] Speaker 13: what happens. You know, is Anthropic's next release of whatever
[26:11] Speaker 13: it is, Anthropic for financial services, going to take the
[26:13] Speaker 13: place of a lot of software companies? So I think
[26:16] Speaker 13: that's already happened. I don't think it's really a result
[26:18] Speaker 13: of guardrails or lack of guardrails. But we are seeing,
[26:21] Speaker 13: for example, a lot of our portfolio companies now starting
[26:24] Speaker 13: to question and realize what the actual cost of this
[26:26] Speaker 13: usage is.
[26:27] Speaker 1: And the honest answer is we don't know.
[26:29] Speaker 13: We know that it's going up consistently, but we don't
[26:31] Speaker 13: know what kind of there is no end game.
[26:33] Speaker 1: So we don't know where it's going to be headed.
[26:36] Speaker 6: There's a really big focus right now on the absolute frontier, right?
[26:41] Speaker 6: That's just fact. The most capable. But what about AI
[26:46] Speaker 6: at the other end? What about AI for everyday people
[26:49] Speaker 6: who don't need it to be spectacular?
[26:50] Speaker 5: Is there an opportunity in that?
[26:53] Speaker 1: Absolutely, there is 100%.
[26:54] Speaker 13: And I think we focus a lot on the front
[26:57] Speaker 13: runners and the most expensive versions which have the best capabilities.
[27:01] Speaker 13: But the reality is on a day-to-day basis, whether it's
[27:03] Speaker 13: an end user like you or me or our portfolio companies,
[27:06] Speaker 13: there are a lot of use cases that don't necessarily
[27:09] Speaker 13: require the most expensive version of models. And to be honest,
[27:12] Speaker 13: this is where, for example, things like DeepSeq come into play,
[27:16] Speaker 13: really low-cost models for everyday, day-to-day tasks.
[27:19] Speaker 1: But there is a question.
[27:20] Speaker 13: Mark around the security level and whether policy is going
[27:23] Speaker 13: to allow us to use some of those types of models.
[27:25] Speaker 13: But 100%, there are different capabilities, different models for different
[27:29] Speaker 13: use cases, and that does help keep the cost down
[27:31] Speaker 13: for sure.
[27:32] Speaker 6: Karen, you're coming to us from Boston. but you're really
[27:35] Speaker 6: focused on the UK and Europe.
[27:38] Speaker 5: So explain that.
[27:39] Speaker 6: Part of it, but everything we've discussed, how is that
[27:42] Speaker 6: playing out in that market different to the United States?
[27:47] Speaker 1: It's a really good question.
[27:47] Speaker 13: So as you mentioned, Beringi is both US and European investors.
[27:51] Speaker 13: So half of our portfolio is in the US, half
[27:53] Speaker 13: is in Europe. We are definitely finding that Europe, which
[27:56] Speaker 13: is not unusual, is slightly ahead of the curve from
[27:58] Speaker 13: a regulatory and safety perspective. There's more legislation that has
[28:02] Speaker 13: already happened and there's a bit more forefront of asking
[28:06] Speaker 13: about the AI safety regulations. I'm a co-founder of a
[28:09] Speaker 13: business called Reframe Ventures, which is effectively looking at both
[28:12] Speaker 13: ESG and safety requirements for AI into early stage companies.
[28:17] Speaker 13: And the Reframe Ventures framework has been adopted widely within Europe.
[28:21] Speaker 13: And I would say there's just a more, I guess,
[28:24] Speaker 13: forward view of what are we looking at, what are
[28:26] Speaker 13: the risks and controls that we need to be looking
[28:27] Speaker 13: for within the European ecosystem, although it's absolutely increasing in
[28:31] Speaker 13: the US. And we are finding that in the US,
[28:34] Speaker 13: whether it's limited partners looking at us for investment or
[28:38] Speaker 13: investors looking at our portfolio for investment, there are a
[28:40] Speaker 13: lot more questions now about AI safety and regulation within
[28:43] Speaker 13: the companies.
[28:45] Speaker 6: Karen McCormick, Chief Investment Officer at Beringia. It's really great
[28:48] Speaker 6: to have you back on Bloomberg Tech. Thank you very
[28:50] Speaker 6: much for your time. Let's get out to New York.
[28:52] Speaker 6: Bloomberg's Yohaira Anand has all the other news.
[28:54] Speaker 5: Hi, Yohaira.
[28:55] Speaker 14: Hi, Ed.
[28:55] Speaker 15: It's time now for Talking Tech. First up, a group
[28:58] Speaker 15: of 10 banks is lining up a $ 22 billion loan
[29:01] Speaker 15: for Blackstone and Alphabet's new cloud venture Crux AI. That's
[29:06] Speaker 15: today's big number. The money will fund the purchase of
[29:09] Speaker 15: of Google's AI chips with the chips themselves and customer
[29:12] Speaker 15: contracts backing the debt. It's the latest mega financing deal
[29:16] Speaker 15: fueling that AI infrastructure boom. Plus, Huawei is turning up
[29:20] Speaker 15: the heat on Nvidia. The Chinese tech giant is moving
[29:23] Speaker 15: up the launch of its next generation Ascend AI chips
[29:27] Speaker 15: by several months.
[29:29] Speaker 1: To early 2027.
[29:29] Speaker 15: It's racing to replace Nvidia in China and compete globally. Plus,
[29:35] Speaker 15: Lucid CEO Silvio Napoli says the EV maker is working
[29:39] Speaker 15: with Bolt to develop an autonomous mobility platform in Europe
[29:43] Speaker 15: with a launch expected later this year. Here's what he
[29:46] Speaker 15: had to say earlier today.
[29:48] Speaker 2: This is a demonstration of our ability to execute in
[29:54] Speaker 2: delivering on using our technology platform, not only in exceptional
[29:59] Speaker 2: electric vehicles, but also in other technologies starting with Robotaxi.
[30:04] Speaker 2: To your question about exclusivity, as you know, we have
[30:06] Speaker 2: an ongoing project with Uber, extremely important, which is due
[30:12] Speaker 2: to launch either end this year or beginning next year.
[30:16] Speaker 2: So there is no exclusivity, but it's really about building
[30:19] Speaker 2: our platform and our experience and having two companies joining together.
[30:25] Speaker 5: Big thank you to Yohaira.
[30:26] Speaker 6: Now coming up in the program, AI pioneer Andrew Ng
[30:29] Speaker 6: gives us his take on the AI Duma worries. This
[30:32] Speaker 6: is a big name in AI research and has been
[30:35] Speaker 6: for a very long time.
[30:36] Speaker 5: This is Bloomberg Tech.
[30:49] Speaker 12: The U.S.
[30:49] Speaker 14: Will be ahead in many things, such as enterprise AI,
[30:53] Speaker 14: making money, building breakthrough.
[30:55] Speaker 5: Technologies, models.
[30:57] Speaker 14: And China will be ahead in other things, maybe consumer applications,
[31:01] Speaker 14: AI devices. So I think there will be plenty of
[31:06] Speaker 14: room for both countries to win.
[31:09] Speaker 6: That was Kai-Fu Lee, CEO of O1. AI and Chairman of
[31:13] Speaker 6: Synovation Ventures. This geopolitical battle to win the AI race
[31:16] Speaker 6: has been thrown into a different light amid the growing
[31:19] Speaker 6: debate over the pacing of the technology's frontier. Joining us
[31:23] Speaker 6: is Andrew Ng, a pioneering AI researcher and founder of
[31:25] Speaker 6: the Google Brain Team, who's now the Managing General Partner
[31:28] Speaker 6: at AI Fund and the founder of DeepLearning.AI and a
[31:32] Speaker 6: longtime adjunct professor of computer science at Stanford. And you've
[31:35] Speaker 6: long said the AI Duma calls are just leading companies
[31:39] Speaker 6: to try and preserve their lead. I would just love
[31:41] Speaker 6: you to weigh in at this moment, Professor Ng.
[31:44] Speaker 16: Now, great to be back on the show, Ed. About
[31:47] Speaker 16: three years ago, as AI was taking off, I was
[31:49] Speaker 16: really surprised by the intensity of the hype or the
[31:53] Speaker 16: stoking fears about AI for publicity purposes to get people
[31:57] Speaker 16: to pay attention or to convince lobbyists to pass regulations
[32:00] Speaker 16: to shut up competitors. Over the last three years, a
[32:03] Speaker 16: lot of those arguments that AI could lead to human
[32:05] Speaker 16: extinction fortunately lost credibility when people went to Congress to
[32:10] Speaker 16: testify when we really looked at it. I was quite
[32:12] Speaker 16: dismayed when over the last two weeks, because of new
[32:15] Speaker 16: advances in AI, but nothing that dangerous, in my opinion,
[32:19] Speaker 16: this wave of PR has kicked up again for probably
[32:22] Speaker 16: similar purposes as before, maybe some new ones. And I
[32:25] Speaker 16: think this is very damaging. We need to innovate and
[32:28] Speaker 16: use AI for many of the wonderful things we could do,
[32:31] Speaker 16: It's not without problems. There are some risks. I think
[32:33] Speaker 16: cybersecurity is one thing to pay a lot of attention to.
[32:36] Speaker 16: But this recent stilt of fear about AI leading to
[32:39] Speaker 16: human extinction and so on is much more science fiction
[32:42] Speaker 16: than science. It's very damaging.
[32:45] Speaker 6: There was a disclosure from OpenAI this morning of several
[32:48] Speaker 6: incidences of models misbehaving, and they outlined them. They did
[32:52] Speaker 6: not include the hacking of third parties. And OpenAI explained
[32:56] Speaker 6: their new policy for what they'll do in disclosing those
[32:59] Speaker 6: kinds of incidents. But lower down the post, they basically
[33:02] Speaker 6: were pretty clear, Andrew, that alignment and safety and monitoring
[33:09] Speaker 6: are not at a sufficient level right now. From a
[33:12] Speaker 6: researcher's perspective, why? has alignment and safety not kept up?
[33:17] Speaker 16: So when any technology develops, our ability to control it
[33:22] Speaker 16: at the outset is limited. We have to try it
[33:24] Speaker 16: out in a safe environment, see where it goes wrong,
[33:26] Speaker 16: and then corral it. I'm going to make an analogy
[33:29] Speaker 16: to the invention of airplanes. No one could control, the
[33:32] Speaker 16: Wright brothers couldn't control an airplane perfectly. And in the
[33:35] Speaker 16: early days, airplanes crashed. Sometimes people died. It was really tragic.
[33:39] Speaker 16: But we learned from those, and we learned to control
[33:41] Speaker 16: airplanes much better. To this day, no one can perfectly
[33:44] Speaker 16: control an airplane. It still gets buffeted around by winds.
[33:47] Speaker 16: But for the most part, we can step in an
[33:49] Speaker 16: airplane and, you know, mostly not fear for our lives.
[33:51] Speaker 16: And I think AI, in an early stage of development,
[33:54] Speaker 16: cannot be controlled perfectly. And I don't think we'll ever
[33:57] Speaker 16: control it perfectly. It will always seem a little bit unpredictable,
[34:01] Speaker 16: just like airplanes today are a little bit unpredictable. But
[34:03] Speaker 16: if we go through this engineering process of corralling AI,
[34:07] Speaker 16: we'll make a lot of progress. I want to give
[34:09] Speaker 16: you one reason why I'm optimistic about our ability to
[34:11] Speaker 16: control AI. If you look at text on the internet,
[34:15] Speaker 16: a lot of it, some of it is racist, sexist,
[34:17] Speaker 16: and so on. And so initially, AI learned to demonstrate
[34:21] Speaker 16: those type of racist or sexist and so on values.
[34:24] Speaker 16: But it turns out our tools for making AI non-racist
[34:28] Speaker 16: are so much better than our tools for convincing any
[34:31] Speaker 16: racist human to be less racist. We can go in,
[34:33] Speaker 16: zero out a few numbers, change a few numbers, and
[34:35] Speaker 16: suddenly the AI is much less racist. And so there
[34:39] Speaker 16: exists work to be done. I'm glad that Opinion and
[34:42] Speaker 16: others are improving transparency. But rather than feeling like it's
[34:45] Speaker 16: the end of the world, we can't solve these. These
[34:47] Speaker 16: are practical engineering problems that we can work on and
[34:50] Speaker 16: that I think we'll make a lot of progress on.
[34:52] Speaker 6: So I'm listening to your answer, Andrew, and the question
[34:55] Speaker 6: that comes to mind is who is responsible ultimately? That's
[34:57] Speaker 6: been the debate of the last week, right? You know,
[35:00] Speaker 6: for many, there was pushback against the frontier labs who
[35:04] Speaker 6: said we can only do this as a coalition. If
[35:07] Speaker 6: one of us does it in isolation, it won't work.
[35:10] Speaker 6: What do you think is the right mechanism?
[35:13] Speaker 16: I think that, so I think that if I do
[35:17] Speaker 16: something on my computer and my computer, does something bad,
[35:20] Speaker 16: there's a question of liability. Who should be responsible? Maybe,
[35:24] Speaker 16: so again, make an analogy. It's an everyday thing. If
[35:27] Speaker 16: I use a hammer and, you know, damage someone else's property,
[35:31] Speaker 16: probably I am liable as a user of the tool,
[35:34] Speaker 16: unless the hammer is so badly made that when I
[35:37] Speaker 16: use the hammer, the head flies off and hits something.
[35:40] Speaker 16: So I think tools should be made to some reasonable standard.
[35:44] Speaker 16: But then so long as the tools, you know, were
[35:47] Speaker 16: built using reasonable diligence and reasonable care, then I think
[35:50] Speaker 16: for the most part, it's the wielder of the tool
[35:52] Speaker 16: that's responsible for how they use it. AI is an
[35:54] Speaker 16: amazing tool. It is a normal technology. It's an amazing technology.
[35:58] Speaker 16: But I think a lot of the time, so long
[35:59] Speaker 16: as the tool is built with reasonable care and diligence,
[36:02] Speaker 16: it's a question of how we use it. Take the
[36:05] Speaker 16: open AI hugging face hack incident. I think a lot
[36:07] Speaker 16: of that was open AI, unfortunately, had insufficient protections and
[36:11] Speaker 16: sandboxing guardrails. To improve that, that wouldn't have happened.
[36:17] Speaker 5: I appreciate.
[36:18] Speaker 6: Professor, you're not inside the Frontier Labs necessarily, but for example,
[36:23] Speaker 6: one of the disclosures of OpenAI this morning in Models
[36:25] Speaker 6: Misbehaving was a model uploading a document to the web
[36:31] Speaker 6: so that it could use it as a citation, as
[36:33] Speaker 6: evidence in its answer. Do you think that we have
[36:36] Speaker 6: a firm understanding of why the models do that from
[36:39] Speaker 6: an alignment perspective, make those actions that are just outside
[36:43] Speaker 6: the scope of instructions?
[36:45] Speaker 16: Yeah, I did see that example. I thought it was,
[36:48] Speaker 16: in some ways, very clever. And the open-eyed disclosure was interesting.
[36:52] Speaker 16: For those who have a lot of friends in the
[36:54] Speaker 16: Frontier Labs, actually, I think I'm the only person in
[36:57] Speaker 16: the world that probably both Sam and Dario has worked for.
[37:00] Speaker 16: So I actually know these teams well. And models are
[37:04] Speaker 16: very clever and will sometimes find ways that are surprising.
[37:09] Speaker 16: But the normal engineering practice is to build these things
[37:12] Speaker 16: test them out in a safe container environment with good sandboxing,
[37:16] Speaker 16: good guardrails to understand the ways it goes wrong so
[37:19] Speaker 16: you can fix them. One of my worries about the
[37:22] Speaker 16: blanket calls to slow down AI is that will actually
[37:25] Speaker 16: slow down the fixes as well. And I think, you know,
[37:28] Speaker 16: it's interesting. When various models were released, I remember when
[37:32] Speaker 16: GPT-2 was released, there were people at some of the
[37:36] Speaker 16: frontier labs saying, can't release GPT-2, this is just way
[37:39] Speaker 16: too dangerous. Guess what? Today, there are open-weight models that
[37:43] Speaker 16: anyone can download and use for free that are way
[37:45] Speaker 16: more capable than GPT-2 or GPT-3 or even GPT-4, and
[37:49] Speaker 16: the world has not ended. So it's interesting how with
[37:52] Speaker 16: every wave of technical innovation, there's this line of hype
[37:55] Speaker 16: that this is so dangerous. But then when it actually
[37:57] Speaker 16: becomes widely available, all of these previous lines of hype
[38:02] Speaker 16: have proven not to be true. But somehow people keep
[38:05] Speaker 16: on buying into this argument that, well, maybe they were
[38:07] Speaker 16: wrong the last five times in a row, but what
[38:09] Speaker 16: if they're right this one time?
[38:11] Speaker 6: You are a respected and decorated researcher and academic in
[38:16] Speaker 6: this field, right? But you are also an investor and
[38:19] Speaker 6: a founder in industry. And so I just want the
[38:23] Speaker 6: audience to have that context. But there's a lot of
[38:26] Speaker 6: the discussion here about how the industry has presented AI
[38:30] Speaker 6: to everyday people. They have anxiety and fear in part
[38:34] Speaker 6: because of the communication coming from you to them.
[38:38] Speaker 5: Speak to that.
[38:39] Speaker 16: Yeah, actually, I hope not me personally, but yes, the
[38:43] Speaker 16: industry has a very problematic set of messages. I'm extremely
[38:47] Speaker 16: bullish about the applications we can build AI to help businesses,
[38:52] Speaker 16: help people. In the US, it is very unfortunate that
[38:56] Speaker 16: starting around 2023, I think for a mix of publicity, hype,
[39:02] Speaker 16: maybe fundraising, maybe trying to lobby regulators to pass regulations
[39:06] Speaker 16: to stifle competitors. a number of individuals and businesses have
[39:10] Speaker 16: come out to stoke fears about AI. So I think
[39:14] Speaker 16: back in 2023, there was a really irresponsible analogy made
[39:18] Speaker 16: between AI and nuclear weapons. I mean, one is a
[39:21] Speaker 16: form of intelligence that helps us think about things, the
[39:24] Speaker 16: other blows up cities. I just don't understand any basis
[39:26] Speaker 16: for the analogy. But this repeated negative sentiment has made
[39:30] Speaker 16: most Americans hate AI. And this is very unfortunate because
[39:36] Speaker 16: one of the, imagine if we convince people to not
[39:39] Speaker 16: use electricity. Electricity has problems. It generates pollution and so on.
[39:44] Speaker 16: But frankly, all of our lives are so much better
[39:46] Speaker 16: off that we have electricity in our lives. And today,
[39:49] Speaker 16: I see high school students wondering, huh, maybe they don't
[39:52] Speaker 16: want to use AI because it could lead to human extinction.
[39:55] Speaker 16: And this will slow down American AI development. There's also
[40:00] Speaker 16: evidence that foreign powers have been running influence operations to
[40:03] Speaker 16: try to slow down America and other democratic nations' sentiment
[40:07] Speaker 16: toward AI. Because if we slow down building data centers,
[40:11] Speaker 16: which is a key infrastructure for the future.
[40:15] Speaker 6: It's a subject, Professor Ng, that we've talked about on
[40:18] Speaker 6: the program a lot, the data center pushback.
[40:21] Speaker 5: I'm very sorry, we were out of time.
[40:22] Speaker 6: Andrew Ng, founder of Deep Learning AI, managing partner at
[40:25] Speaker 6: AI Fund, and adjunct professor at Stanford. Some breaking news
[40:28] Speaker 6: on the Bloomberg terminal. Elon Musk's SpaceX. has held internal
[40:32] Speaker 6: discussions about buying customer and operational information data from troubled
[40:39] Speaker 6: or defunct startups, according to sources. It's targeting a more
[40:42] Speaker 6: affordable data source to improve its AI models. The people
[40:46] Speaker 6: said the talks within teams inside SpaceX AI are informal
[40:50] Speaker 6: and may not result in deals. As the global AI
[40:54] Speaker 6: debate heats up, another major AI-related discussion is roiling the U.S.
[40:58] Speaker 5: Tech capital.
[40:59] Speaker 6: Who is reaping the financial benefits of the AI boom,
[41:02] Speaker 6: and who's missing out, especially as tech workers in Silicon
[41:05] Speaker 6: Valley face layoffs? Bloomberg's California reporter Francesca Maglioni joins us now.
[41:09] Speaker 5: That's the story.
[41:10] Speaker 6: There is a very big group of technology industry people
[41:13] Speaker 6: that are missing out here on a wealth boom.
[41:16] Speaker 5: What are you reporting?
[41:17] Speaker 4: Yeah, I reported that about twice as many layoffs happened
[41:20] Speaker 4: in the tech industry this year. I talked to many
[41:23] Speaker 4: tech workers that were part of the big tech boom
[41:25] Speaker 4: a couple years ago and are now finding themselves struggling
[41:28] Speaker 4: to find new jobs and trying to understand how their
[41:31] Speaker 4: skills fit into this new labor market.
[41:32] Speaker 6: Part of this is the types of jobs that are
[41:34] Speaker 6: currently in favor, very related to AI, and types of
[41:38] Speaker 6: jobs that are not, not as related to AI.
[41:40] Speaker 5: Tell me about that.
[41:41] Speaker 1: Yeah, a lot of these people are coders.
[41:43] Speaker 4: They got these jobs when they just learned a couple
[41:45] Speaker 4: languages and that was the best skill you could have
[41:47] Speaker 4: a couple years ago. And now those jobs are mostly automated.
[41:50] Speaker 4: And so all these new jobs have AI tied next
[41:53] Speaker 4: to it.
[41:53] Speaker 1: And a lot of.
[41:54] Speaker 4: People are confused as to just what those skills mean
[41:56] Speaker 4: and what you have to know to be favorable.
[41:58] Speaker 6: Let's very quickly get to the missing out bit. So
[42:00] Speaker 6: a lot of these people are still searching for jobs, right?
[42:02] Speaker 6: They're not working at one of the labs that might
[42:04] Speaker 6: go public, but they want to be a part of it.
[42:07] Speaker 5: Tell me about that.
[42:08] Speaker 1: Yeah, I mean, it's very hard. Every day there's a
[42:10] Speaker 1: new model.
[42:11] Speaker 4: Every day there's new skills that they need to learn.
[42:13] Speaker 4: And so these people are trying to catch up, but
[42:15] Speaker 4: it's really hard to kind of keep up with the moment.
[42:17] Speaker 5: What were some of the data points in the story?
[42:20] Speaker 4: Yeah, some of the data points that were interesting was
[42:21] Speaker 4: that younger workers are especially vulnerable to AI relative companies,
[42:26] Speaker 4: and their skills are kind of harder to match with
[42:29] Speaker 4: this new labor market.
[42:30] Speaker 5: Okay, I really recommend you go and read this one.
[42:31] Speaker 6: It's really resonating on Bloomberg platforms. Bloomberg's Francesca Maglioni, thank
[42:34] Speaker 6: you very much. That does it for this edition of
[42:37] Speaker 6: Bloomberg Tech. Recap the show. We've had a lot of
[42:40] Speaker 6: very high-level conversations about what is happening in AI right now.
[42:43] Speaker 6: You know where to find the podcast. It's on all
[42:45] Speaker 6: Bloomberg platforms and online on Apple.com. Spotify, and iHeart.
[42:49] Speaker 5: It is Thursday.
[42:50] Speaker 6: There is one more day to go in what has
[42:52] Speaker 6: been a big week in the world of tech.
[42:54] Speaker 5: Thanks for watching. This is Bloomberg Tech.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.