# AI’s Biggest Players Debate a Frontier Slowdown

An episode of [Bloomberg Tech](https://invisiblenetwork.ai/podcast/1611767434), 14 Sep 2026, 43 min.

Audio: https://podtrac.com/pts/redirect.mp3/traffic.omny.fm/d/clips/e73c998e-6e60-432f-8610-ae210140c5b1/41764a4f-fc64-4e11-89ba-ae7c0030ab5e/261f3ea9-dfc8-4471-af2e-b4c501209f1b/audio.mp3?utm_source=Podcast&in_playlist=9caafc41-289c-4115-995d-ae7c0030ab75

Bloomberg’s Ed Ludlow breaks down calls from AI for a slower pace at the frontier, arguing safety needs time to catch up with increasingly powerful models. The debate is reaching Washington and Wall Street, with AI-linked stocks under pressure as investors weigh what slower development could mean for the spending boom. See omnystudio.com/listener for privacy information.

## Transcript

[0:02] Speaker 1: Bloomberg Audio Studios.
[0:04] Speaker 2: Podcasts.
[0:05] Speaker 3: Radio.
[0:06] Speaker 4: News.
[0:13] Speaker 1: Bloomberg Tech is live from the heart of Silicon Valley
[0:17] Speaker 1: with Ed Ludlow in San Francisco.
[0:22] Speaker 5: This is Bloomberg Tech. AI leaders are calling for a
[0:25] Speaker 5: slower pace at the frontier. Anthropic CEO Dario Amadei saying
[0:29] Speaker 5: over the weekend, quote, we must slow the pace at
[0:31] Speaker 5: which we improve the capabilities of AI models. His argument,
[0:35] Speaker 5: safety needs time to keep up. OpenAI's Sam Altman and
[0:39] Speaker 5: SpaceX AI's Elon Musk quickly backing the idea.
[0:42] Speaker 1: Anthropic's now opening.
[0:43] Speaker 5: Its doors to independent evaluators, while Altman says OpenAI will
[0:48] Speaker 5: not go public this year as it prioritizes safety. This
[0:53] Speaker 5: has prompted a response from President Trump this morning, saying
[0:55] Speaker 5: in a True Social post that his administration has, quote,
[0:58] Speaker 5: stopped AI people from doing bad things and they will
[1:01] Speaker 5: continue to do so. And the debate's hitting markets, NVIDIA, Micron,
[1:05] Speaker 5: other AI-linked stocks all under pressure as investors ask what
[1:08] Speaker 5: a slower frontier could mean for the AI spending boom.
[1:12] Speaker 5: Throughout the hour, what pacing actually means, the industry response,
[1:16] Speaker 5: and what comes next. From Washington, Bloomberg's AI reporter Rachel
[1:20] Speaker 5: Metz And tech editor in D.C., Mike Shepard, join us
[1:23] Speaker 5: for the details. Rachel, let's start with the essay from
[1:25] Speaker 5: Anthropic CEO Dario Amadei, which was published this Saturday.
[1:28] Speaker 1: What do we need to know?
[1:29] Speaker 6: I mean, he's talking a lot about some of the
[1:32] Speaker 6: wonderful things that AI can lead to, but also how
[1:35] Speaker 6: recent events such as an inadvertent hack by open AI
[1:39] Speaker 6: models of hugging face are leading to this moment where
[1:43] Speaker 6: he believes that some sort of pacing is necessary in
[1:46] Speaker 6: order to ensure safety of AI.
[1:49] Speaker 5: Shep, you were on hand all weekend on duty to
[1:51] Speaker 5: basically bring together all of the responses that came in. First,
[1:55] Speaker 5: it was Sam Altman, then Elon Musk, and then lots
[1:57] Speaker 5: of investors and politicians weighed in, too.
[2:00] Speaker 1: Try and summarize it.
[2:02] Speaker 7: Well, you know, quickly, we saw the arch rivals of Anthropic,
[2:08] Speaker 7: Sam Altman from OpenAI, Elon Musk from SpaceX AI, quickly
[2:12] Speaker 7: embrace in broad terms the Dario Amadei's call for a slowdown.
[2:16] Speaker 7: When you get into the specifics, it's a little bit
[2:19] Speaker 7: unclear how closely aligned they all are.
[2:22] Speaker 1: But on the broad strokes, they back it.
[2:24] Speaker 7: Investors are a little bit skeptical about how far the
[2:27] Speaker 7: companies will actually go in slowing this down. And for
[2:31] Speaker 7: their part, Dario Amadei and Sam Altman are insisting that
[2:34] Speaker 7: slowing the pace at the frontier does not mean stopping
[2:38] Speaker 7: spending or slowing down work or growth in their businesses.
[2:42] Speaker 5: There is no commitment to end training runs or reduce spending.
[2:46] Speaker 1: I think that's really important.
[2:47] Speaker 5: Rachel, I think it's worth going back over the timeline
[2:50] Speaker 5: of the last seven days. When we started this story,
[2:54] Speaker 5: it was because an anthropic researcher had resigned. And we
[2:57] Speaker 5: also had reporting that Sam Altman, who's the CEO of OpenAI,
[3:01] Speaker 5: had already gone company-wide to say, we think we should
[3:04] Speaker 5: slow the pace, fill the gaps.
[3:06] Speaker 1: Okay.
[3:07] Speaker 6: So we had this resignation.
[3:09] Speaker 1: It seemed to gain ground.
[3:11] Speaker 6: It gained a lot of attention, not just within AI,
[3:15] Speaker 6: but also outside of AI. And then Sam Altman, as
[3:17] Speaker 6: you mentioned, our reporting indicated that he had said that
[3:21] Speaker 6: the company was interested. um, in pacing, um, and was
[3:26] Speaker 6: hoping other companies would join as well. And then since then,
[3:29] Speaker 6: we're just seeing, um, this sort of snowball effect where
[3:33] Speaker 6: we're getting more and more talking toward this idea of pacing.
[3:38] Speaker 6: But as you guys both mentioned, it's not.
[3:41] Speaker 3: Clear what that would mean.
[3:42] Speaker 6: And it to actually slow down innovation internally, perhaps more externally.
[3:48] Speaker 6: And also they're talking about using some outside companies as
[3:52] Speaker 6: well to look over their models and that kind of thing.
[3:55] Speaker 6: So a lot of unanswered questions still, I would say.
[3:58] Speaker 1: We've had the reaction from the president this morning.
[4:00] Speaker 5: Over the weekend, the reaction from Congress, Democratic Senator Chris
[4:03] Speaker 5: Coons of Delaware speaking to Bloomberg this weekend.
[4:06] Speaker 8: Now that we've got agreement from Elon Musk and Sam
[4:09] Speaker 8: Altman and Dario Amadei, we need agreement from Republicans and
[4:13] Speaker 8: Democrats in Congress to put aside our partisan pre-election bickering
[4:18] Speaker 8: and put in place some real guardrails before AI goes
[4:22] Speaker 8: off the rails.
[4:24] Speaker 1: Shep, what's happening on Capitol Hill right now? Well, there's
[4:27] Speaker 1: a lot of talk.
[4:28] Speaker 7: And we heard Chris Coons really lay out his vision
[4:31] Speaker 7: for trying to get both parties together quickly before the
[4:35] Speaker 7: election to find some consensus on this. But in a
[4:37] Speaker 7: bitterly divided House and Senate, it's hard to imagine that
[4:41] Speaker 7: they would be able to agree on something that's so
[4:43] Speaker 7: technically difficult to wrap their heads around. In that interview,
[4:47] Speaker 7: Coons also pointed to a relative lack of tech expertise
[4:51] Speaker 7: on Capitol Hill. And then meanwhile, going down Pennsylvania Avenue
[4:55] Speaker 7: inside the Trump administration, we heard The president just now
[4:58] Speaker 7: in social media really going after Dario Amadei directly over
[5:02] Speaker 7: this call for guardrails, insisting that he as president is
[5:05] Speaker 7: the best guardrail of all. And, you know, you look
[5:09] Speaker 7: at this, stepping back for a moment, AI development is
[5:13] Speaker 7: central to the president's economic agenda. And it's also critical
[5:17] Speaker 7: to this upcoming meeting with Chinese leader Xi Jinping. The president,
[5:21] Speaker 7: Donald Trump, has put AI at the center of the
[5:25] Speaker 7: economic competition with China. And he doesn't want anything holding
[5:28] Speaker 7: the U.S. back in this moment.
[5:31] Speaker 5: Bloomberg's Rachel Metz here in San Francisco, Mike Shepard in Washington, D.C.
[5:34] Speaker 5: Thank you very much. The AI trade's under pressure as
[5:37] Speaker 5: investors are asking whether slowing AI development could also slow
[5:40] Speaker 5: the massive spending boom behind it. Uday Thiruvu, portfolio manager
[5:44] Speaker 5: at Harding Lova, joins us now, and a portfolio manager
[5:47] Speaker 5: who's focused on some of the names being hit hardest
[5:49] Speaker 5: this morning, NVIDIA on the compute side, the memory names,
[5:52] Speaker 5: some of the Neo clouds. Uday, just start with your
[5:55] Speaker 5: reaction to the weekend's events as somebody that is directly
[5:58] Speaker 5: invested in this corner of the technology market.
[6:01] Speaker 9: So I think the weekend's events sets in play that
[6:03] Speaker 9: it's not just about demand and demand growth, but realistic
[6:07] Speaker 9: timeline for this and how implementation of AI is really
[6:11] Speaker 9: going to happen. We're going from the face of super
[6:14] Speaker 9: optimistic blue sky to, hey, hang on, there's a few
[6:17] Speaker 9: more hurdles here that we need to get through. And
[6:19] Speaker 9: I think that's what investors are grappling with at this
[6:21] Speaker 9: point in time is how much of this current pacing
[6:25] Speaker 9: is going to impact the demand environment over the next two,
[6:27] Speaker 9: three years, but also over.
[6:29] Speaker 1: The longer term.
[6:32] Speaker 5: There is a debate about what pacing actually means, right?
[6:35] Speaker 5: There have been many voices. We're going to go to
[6:37] Speaker 5: one of them later in the program, Greg Brockman, OpenAI president,
[6:40] Speaker 5: saying that all the open source work continues. It is
[6:43] Speaker 5: the absolute frontier cutting edge. As an investor in the space,
[6:48] Speaker 5: is it linear that if they were to slow down
[6:50] Speaker 5: at the frontier, they would reduce spending in aggregate on
[6:54] Speaker 5: the compute needed?
[6:55] Speaker 9: No, absolutely not. Because if you think about it, most
[6:58] Speaker 9: of the frontier models are not being used on a
[7:01] Speaker 9: wide spaces now anyway. If you talk to most enterprises,
[7:04] Speaker 9: they're looking at how to basically optimize their costs by
[7:07] Speaker 9: using some open source models, some of the older models.
[7:10] Speaker 9: And maybe for extremes and tasks that require a lot
[7:13] Speaker 9: of reasoning, the latest model. So what this means is
[7:16] Speaker 9: that slowing down the frontier doesn't actually lead to a
[7:20] Speaker 9: slowdown of the total demand curve, but just basically spreads
[7:23] Speaker 9: the curve out over a longer period of time. To us,
[7:25] Speaker 9: that's what investors would need to focus their minds on
[7:28] Speaker 9: is this is a timing issue as opposed to a
[7:31] Speaker 9: total scale of demand issue.
[7:34] Speaker 5: There was one response from David Sachs who said, okay,
[7:38] Speaker 5: if you think you, the Frontier Labs, should slow the
[7:40] Speaker 5: pace at the frontier, just do it. You're in control
[7:43] Speaker 5: of your own destiny. And the sort of other side
[7:46] Speaker 5: of that coin is the need for government to come
[7:48] Speaker 5: in and regulate. You've got a very interesting point of
[7:50] Speaker 5: view on the Frontier Labs and the need for regulation.
[7:54] Speaker 9: Yeah, look, from a Frontier Labs perspective, there's two things, right? Firstly,
[7:57] Speaker 9: they're talking about how The technology can be used for
[8:01] Speaker 9: dangerous purposes and therefore regulation needs to come in because
[8:04] Speaker 9: if one frontier lab allows them to do it and
[8:06] Speaker 9: the other doesn't, the one who doesn't is at a
[8:08] Speaker 9: competitive disadvantage. So that's why what they want is a
[8:11] Speaker 9: set of rules that applies to everyone. So therefore no
[8:14] Speaker 9: one has an incentive or a disincentive to slow down
[8:17] Speaker 9: or speed up.
[8:18] Speaker 1: So that's the first one.
[8:19] Speaker 9: But then also the second aspect of this is if
[8:21] Speaker 9: there are regulations in place, then the companies who are
[8:24] Speaker 9: already ahead, the Frontier Labs and OpenAI, as well as Anthropic, Gemini,
[8:29] Speaker 9: all of them, they have a strategic advantage because anyone
[8:33] Speaker 9: else behind them has to go at a much slower
[8:35] Speaker 9: pace than they do. So from both these perspectives, what
[8:38] Speaker 9: they want is rules across to everyone, which stops everyone
[8:41] Speaker 9: and also benefits them. So from my perspective, there's a
[8:45] Speaker 9: bit of self-serving from the Frontier Labs that they do
[8:47] Speaker 9: want rules because that is going to help them.
[8:51] Speaker 5: You worried that an anthropic IPO or an open AI
[8:55] Speaker 5: IPO gets delayed, kicked into the long grass because of
[8:58] Speaker 5: the environment we're in?
[9:00] Speaker 9: If you're one of these two companies, you'd want the
[9:02] Speaker 9: revenue environment set and the cost structure set before you
[9:05] Speaker 9: come to an IPO. What you don't want investors is
[9:07] Speaker 9: talking about what the demand curve slowdown might look like,
[9:11] Speaker 9: might not look like. So from that perspective, maybe it
[9:12] Speaker 9: is better for them maybe into next year. But also
[9:15] Speaker 9: the second part of this is What are their returns
[9:18] Speaker 9: looking like right now? We still don't know the profitability
[9:21] Speaker 9: of these companies. They do provide that they're going EBITDA positive,
[9:24] Speaker 9: that they're increasing their profitability. But the pace of that
[9:27] Speaker 9: might not be enough to be able to attract investors
[9:30] Speaker 9: to the level at which that they want. So the
[9:33] Speaker 9: pacing argument actually helps them get a clearer IPO story.
[9:39] Speaker 5: I'm just going to jump in with some other reporting
[9:41] Speaker 5: from the weekend, which came from the Financial Times. The
[9:43] Speaker 5: Anthropic told some investors it was on track for a
[9:46] Speaker 5: second consecutive quarter of profit, where margins are at 80%, 8-0%.
[9:52] Speaker 5: But that doesn't take into account revenue share, which they're
[9:55] Speaker 5: on the hook for with a number of parties, and
[9:57] Speaker 5: also cost of compute, which would seem. to be something
[10:00] Speaker 5: quite big to factor in. How do you interpret that?
[10:04] Speaker 9: So I look at it from the cost of compute
[10:06] Speaker 9: is going up for these companies, because what you're coming
[10:08] Speaker 9: to an environment where if you go back to a
[10:10] Speaker 9: year and a half ago, buying the same servers... is
[10:14] Speaker 9: so much more expensive now. So the next infrastructure that's
[10:18] Speaker 9: coming online comes at a much higher cost. So the
[10:21] Speaker 9: revenue from that must be much higher too. But then
[10:24] Speaker 9: when you look at the top line, that's being affected
[10:26] Speaker 9: by token prices coming down. So we're looking at that
[10:29] Speaker 9: sort of environment where revenue per token is coming down,
[10:32] Speaker 9: but potentially the cost per token And the total depreciation
[10:35] Speaker 9: that comes through is increasing. So maybe what you saw
[10:39] Speaker 9: from 80% is not likely to be the same number
[10:42] Speaker 9: going forward. It could be better or it could be worse,
[10:44] Speaker 9: but we just don't know. And that's where I think
[10:45] Speaker 9: in my perspective, the clarity that comes through over the
[10:48] Speaker 9: next few months as we see this new infrastructure come
[10:51] Speaker 9: online and the earnings and profitability come through, that might
[10:55] Speaker 9: be a better story for Anthropic to be IPO-ing into.
[10:59] Speaker 5: Uday, let's go back to this through the lens of NVIDIA.
[11:01] Speaker 5: We know what's going to happen next year. NVIDIA told
[11:03] Speaker 5: us 70% growth based on supply. Does that change at
[11:08] Speaker 5: all or is this a 2028 story?
[11:10] Speaker 9: No, I think this is a 2028 story because 2027,
[11:13] Speaker 9: most of the companies have bought their demand in. In fact,
[11:16] Speaker 9: there's a significant amount of overordering because companies know that
[11:20] Speaker 9: they need these chips to do the agentic AI that
[11:24] Speaker 9: they want to achieve. Now, The performance of models is improving.
[11:28] Speaker 9: So 2028, maybe there's a different type of infrastructure that's required.
[11:32] Speaker 9: But for 2027, if you hadn't locked in by now,
[11:36] Speaker 9: you are not going to get them. And if you
[11:37] Speaker 9: have them, you're not going to let them out of
[11:39] Speaker 9: your hand. So that's why I think 2027 is most
[11:42] Speaker 9: likely locked in for NVIDIA and put across the whole
[11:44] Speaker 9: supply chain, including the NEO clouds, including everyone. But then
[11:48] Speaker 9: as we get into this time next year, that's when
[11:52] Speaker 9: the pacing, AI regulations... IPOs of these companies, all these
[11:57] Speaker 9: come into play.
[11:58] Speaker 1: But over the next.
[11:59] Speaker 9: 12 months, I don't think we should worry too much
[12:01] Speaker 9: about revenue growth and profitability.
[12:04] Speaker 5: Uday Charuva of Harding Lovner, back on the show. Thank
[12:07] Speaker 5: you very much. Tune in later today. We're going to
[12:10] Speaker 5: speak to David Sachs, Chair of the President's Council of
[12:12] Speaker 5: Advisors on Science and Technology, as he has and continues
[12:16] Speaker 5: to weigh in on what happened over the weekend. One
[12:20] Speaker 5: of Dario Amadei's most concrete proposals for AI safety is
[12:24] Speaker 5: independent oversight, and Fropik says it will bring in third-party
[12:27] Speaker 5: evaluators with near-employee-level access, allowing outsiders to scrutinize not just
[12:33] Speaker 5: finished models, but how they're being developed. Rohan Krishnan is
[12:37] Speaker 5: the founder and CEO of Vows AI. Vows independently tests
[12:40] Speaker 5: and benchmarks AI models. including unreleased models from OpenAI and Anthropic.
[12:45] Speaker 5: He says, embedded evaluators will promote an even playing field
[12:49] Speaker 5: and public trust. And he joins us now. You had
[12:52] Speaker 5: a response to Dario Amadei's essay. Let's start with that response.
[12:55] Speaker 5: What was your conclusion? Yeah, well, I think.
[12:59] Speaker 10: I mean, to be honest, I think there's been a
[13:00] Speaker 10: lot of extreme statements being made over the last week.
[13:03] Speaker 10: And I can understand where they're coming from, but I
[13:06] Speaker 10: think they can also cause undue anxiety for the general public.
[13:09] Speaker 10: And so, really, I think in response to this, I
[13:11] Speaker 10: found reason to be optimistic. From what I see at
[13:14] Speaker 10: Val's AI, boots on the ground, we've actually been able
[13:17] Speaker 10: to see the labs be very forthcoming. And we engage
[13:19] Speaker 10: across all the major foundation model labs, as well as
[13:22] Speaker 10: with members of Congress and government agencies and some of
[13:24] Speaker 10: the largest enterprises. And so I actually think there's a
[13:27] Speaker 10: lot of conflict happening on the public stage, but on
[13:30] Speaker 10: the whole, we have reason to be optimistic that coordinated
[13:32] Speaker 10: effort is possible.
[13:33] Speaker 1: So let's talk about what Valve sees, right?
[13:35] Speaker 5: If we were trying to really simplify what Dario Amodei
[13:38] Speaker 5: and others are calling for, it is to slow down
[13:42] Speaker 5: capability development, the pace of capability development, allowing safety alignment
[13:48] Speaker 5: to catch up. You see these models in the training
[13:51] Speaker 5: phase and before they're released, right?
[13:53] Speaker 10: Yeah, well, I think what we see is that there's
[13:56] Speaker 10: really been an outsized investment towards the generation or the
[13:59] Speaker 10: capabilities of the models without that same parallel investment in
[14:02] Speaker 10: the testing and evaluation. And so our ability to create
[14:05] Speaker 10: better models now outpaces our ability to actually understand them.
[14:08] Speaker 1: That's the issue at hand. A lot of people are asking,
[14:10] Speaker 1: why now?
[14:11] Speaker 5: And what Dario Amodei wrote is about the ability of
[14:15] Speaker 5: AI to aid in the research and development of future
[14:19] Speaker 5: generations of AI, so-called recursive improvement. That was a very
[14:23] Speaker 5: clear piece of the essay. It is something that you
[14:26] Speaker 5: look at at VOWS through the RSI index. So I
[14:29] Speaker 5: think we're going to bring up that index. Explain what
[14:32] Speaker 5: we're looking at, the methodology, and what that means. tells
[14:37] Speaker 5: you about an AI's ability to contribute to the development
[14:40] Speaker 5: of AI.
[14:41] Speaker 1: There it is.
[14:42] Speaker 9: Right.
[14:43] Speaker 10: Well, so this is, yeah, this is the mainline figure
[14:44] Speaker 10: tracking the progress of the major frontier models towards RSI,
[14:47] Speaker 10: recursive self-improvement. And that describes the model's ability to make
[14:52] Speaker 10: the successor version of itself autonomously.
[14:54] Speaker 1: So wholly without human input.
[14:56] Speaker 10: And that's, I think, to contrast today, where models are
[14:59] Speaker 10: used as an active part, but humans are still involved.
[15:02] Speaker 10: And so this would be akin to GPT-6 being responsible
[15:05] Speaker 10: for making GPT-7, or Claude making the next version of
[15:07] Speaker 10: itself without any human researchers. And so what you see
[15:10] Speaker 10: on the plot is that we have... some early signs
[15:13] Speaker 10: that models are exhibiting RSI, but they're still far away
[15:16] Speaker 10: from the human frontier. And in extrapolating out these trends,
[15:19] Speaker 10: we're actually making the prediction that we'll see models eclipse
[15:22] Speaker 10: human researchers probably around August of 2027.
[15:24] Speaker 1: August of 2027.
[15:26] Speaker 5: So the question right now, like loads of people pose
[15:28] Speaker 5: this is, Can an AI model act like an AI
[15:31] Speaker 5: researcher right now? Can it run experiments? Can it improve
[15:35] Speaker 5: AI systems with little or no human help? And right
[15:40] Speaker 5: now the answer is no, but you have line of
[15:42] Speaker 5: sight to when the answer would be yes.
[15:43] Speaker 10: To be clear, we're testing a set of proxies and
[15:46] Speaker 10: the publicly available models. And so the indication that we
[15:49] Speaker 10: see is that the models are actually very good at
[15:50] Speaker 10: executing experiments and operating as engineers. But they lack kind
[15:54] Speaker 10: of the intuition to develop new experiments and have fundamental breakthroughs.
[15:58] Speaker 10: But I think it's also important to add that we're
[16:00] Speaker 10: not testing any unreleased model systems. We're not testing specialized
[16:03] Speaker 10: agents or even multi-agent systems. And so I think the
[16:06] Speaker 10: role of event evaluators over time could be to get
[16:09] Speaker 10: closer to these internal systems.
[16:11] Speaker 5: Again, the kind of concrete proposal from Anthropic is to have...
[16:17] Speaker 5: independent evaluation, but inside the company. It sounds very close
[16:21] Speaker 5: to what you're already doing with that company. This would
[16:24] Speaker 5: create a demand for your services. So how would that work?
[16:27] Speaker 5: And Fropic would pay you. How would that impact your
[16:30] Speaker 5: independence to make an assessment of a yet unreleased model?
[16:34] Speaker 10: Yeah, I think on the whole, it's going to take
[16:36] Speaker 10: a diversity of approaches. So it'd be a failure to
[16:38] Speaker 10: say that this will be the end-all be-all. But I
[16:41] Speaker 10: think there's a historical precedent when you look at things
[16:43] Speaker 10: like ratings agencies or auditing firms who have been for-profit
[16:46] Speaker 10: companies and and have been responsible for testing and evaluation
[16:49] Speaker 10: in other industries, I think there's natural conflicts that arise.
[16:52] Speaker 10: And so the main one that we want to avoid
[16:54] Speaker 10: is a situation where the same group auditing is also
[16:57] Speaker 10: responsible for remediating or fixing. And then you end up
[17:00] Speaker 10: in situations like Enron. And so we've set the line
[17:02] Speaker 10: very clearly where although we make our methodologies public, we
[17:05] Speaker 10: have test sets that are kept private. And we also
[17:08] Speaker 10: ensure that we don't sell training data or the solution
[17:10] Speaker 10: to labs while we do the testing process.
[17:12] Speaker 5: I guess to end our conversation, the obvious next question
[17:15] Speaker 5: is what happens next? You know, people have taken different approaches,
[17:20] Speaker 5: but it seems that this is only going to work
[17:23] Speaker 5: if all of the frontier labs are able to cooperate
[17:25] Speaker 5: with one another to set a standard.
[17:28] Speaker 10: You work with them. Do you see that happening? Honestly,
[17:32] Speaker 10: I have a lot of reason to be optimistic. I
[17:33] Speaker 10: think that people at these labs are rational actors, and
[17:37] Speaker 10: so they see that the way things are headed, it
[17:39] Speaker 10: will take some coordinated effort. And so I think with
[17:42] Speaker 10: or without regulation, I think market-based solutions will have a
[17:45] Speaker 10: part to play. Ryan Krishnan, co-founder and CEO of Vow's AI,
[17:49] Speaker 10: key player in benchmarking testing unreleased models on Bloomberg Tech.
[17:53] Speaker 10: Now coming up, executives may be calling for an AI slowdown,
[17:56] Speaker 10: but the global race to dominate the technology is still
[17:59] Speaker 10: top of mind. For US and Chinese leaders, we have
[18:02] Speaker 10: the latest next. This is Bloomberg Tech.
[18:12] Speaker 4: We're leading China in AI. We're the most sophisticated country
[18:15] Speaker 4: in the world. And frankly, I want to keep it
[18:17] Speaker 4: that way because whoever wins, AI wins. And we can
[18:21] Speaker 4: put guardrails, we can do this and that, but I
[18:23] Speaker 4: think you have a lot of negative forces that are
[18:25] Speaker 4: bringing it up that shouldn't be.
[18:28] Speaker 2: Bringing it up.
[18:30] Speaker 5: President Trump there responding to calls from anthropic Zaria Amodei
[18:33] Speaker 5: and other AI executives to slow the pace of the
[18:36] Speaker 5: technology's development. Chinese leaders also criticized those warnings, with one
[18:41] Speaker 5: official calling them fear-mongering and saying confrontation and vicious competition
[18:46] Speaker 5: would disrupt efforts for global AI governance. Let's get the
[18:49] Speaker 5: latest with Bloomberg Tech Editor. Peter Elstrom. There's a sort
[18:52] Speaker 5: of out-of-order chronology of that. The president was posed the
[18:55] Speaker 5: question after the essay from CEO Dariya Medea of Anthropic,
[19:00] Speaker 5: but then we heard from Chinese officials. Summarize China's position
[19:04] Speaker 5: right now.
[19:06] Speaker 11: So China's position here is quite interesting. The spokesman for
[19:10] Speaker 11: the foreign ministry came out and said the U.S. is
[19:12] Speaker 11: starting this fear-mongering around AI. A couple things to remember.
[19:16] Speaker 11: AI is much more popular with the citizens in China
[19:19] Speaker 11: than it is in the U.S. That's for a number
[19:21] Speaker 11: of different reasons, not coincidentally because... the press is quite
[19:25] Speaker 11: controlled there. But also, the AI companies that are much smaller,
[19:28] Speaker 11: they haven't had the kind of big data center build-outs
[19:30] Speaker 11: that you've seen in the U.S., and they don't have
[19:32] Speaker 11: the kind of power that the American AI companies have.
[19:36] Speaker 1: They're much smaller.
[19:37] Speaker 11: They're not really at the frontier the way we see
[19:39] Speaker 11: with OpenAI and Anthropic. They've been fast followers in this
[19:43] Speaker 11: market so far, and they don't have nearly the kind
[19:45] Speaker 11: of compute that you do have in the US right now.
[19:48] Speaker 11: So the Communist Party has held a pretty tight rein
[19:51] Speaker 11: on the AI companies.
[19:52] Speaker 1: It's also been supporting them.
[19:53] Speaker 11: It's been pushing them towards this open source model that
[19:56] Speaker 11: we've talked about a few times in the past that's
[19:58] Speaker 11: less expensive. It allows companies to adopt the technology a
[20:01] Speaker 11: bit more quickly too. So they're not nearly the big
[20:05] Speaker 11: players in the market that you see.
[20:07] Speaker 1: In the United States.
[20:09] Speaker 5: We've covered in the program so far the idea that
[20:11] Speaker 5: technology stocks, principally chip stocks, are down because if the
[20:16] Speaker 5: pace of AI development slows, maybe the spending slows. But
[20:18] Speaker 5: there's one more directly under pressure stock, and that is SoftBank.
[20:23] Speaker 1: It fell in the Japanese session. Now the U.S.
[20:25] Speaker 5: Listed shares are under pressure after OpenAI's CEO, Sam Altman,
[20:29] Speaker 5: said no IPO this year because of safety concerns. Explain
[20:33] Speaker 5: that SoftBank reaction in the stock.
[20:36] Speaker 11: Yeah, you see this AI ink spill over from the
[20:39] Speaker 11: United States into Asia today. And as the Monday morning
[20:43] Speaker 11: trading started today, SoftBank shares got hit. You also saw
[20:47] Speaker 11: it run into South Korea with SK Hynix and Samsung
[20:50] Speaker 11: also down. SoftBank was down about 11%. Of course, they're
[20:53] Speaker 11: a huge investor in OpenAI. They plan on cashing in
[20:56] Speaker 11: as OpenAI goes public. So As Altman acknowledged that he
[21:00] Speaker 11: was not going to go public this year, he said
[21:02] Speaker 11: it was not the right time for it. That pushes
[21:04] Speaker 11: back the timeline when SoftBank's able to see their holding
[21:08] Speaker 11: develop into a publicly traded stock there. And remember, SoftBank
[21:11] Speaker 11: has a lot on the line here. It's not just
[21:13] Speaker 11: that they're investors. They're actually borrowing money to be able
[21:16] Speaker 11: to put into OpenAI. And they want to also build
[21:19] Speaker 11: out the infrastructure through their Stargate infrastructure. venture to build
[21:23] Speaker 11: up these data centers too. So they have a lot
[21:25] Speaker 11: on the line and the sooner they can get payback,
[21:27] Speaker 11: the better for Masayoshi Son and SoftBank.
[21:30] Speaker 5: Peter, really quick, it became obvious by Saturday, we'd have
[21:33] Speaker 5: a pretty tense Monday open in the US because of
[21:36] Speaker 5: the news headlines and the ideas. What was it like
[21:39] Speaker 5: in London and Asia going through the news flow real quick?
[21:43] Speaker 11: Well, it hit the markets a bit earlier here, as
[21:45] Speaker 11: we were talking about earlier. You definitely saw it with
[21:48] Speaker 11: the memory chip stocks in South Korea, where their shares
[21:51] Speaker 11: were down pretty hard. SK Hynix, which has been the
[21:54] Speaker 11: leader in HBM, was down about 7%. Samsung is down
[21:57] Speaker 11: about 5%, too. So it hit those markets as there
[22:00] Speaker 11: were concerns about maybe the slowdown is going to affect
[22:02] Speaker 11: capital spending. Of course, the companies haven't said that yet,
[22:05] Speaker 11: but you're beginning to see that spill through global markets today.
[22:08] Speaker 1: That's the concern. Bloomberg's Peter Ahlstrom. Thank you very much.
[22:18] Speaker 5: The big story over the weekend, Anthropic CEO Dario Amadei said,
[22:22] Speaker 5: we need to pace the frontier development of AI models.
[22:28] Speaker 5: was supported by Sam Altman, the CEO of OpenAI and
[22:30] Speaker 5: Elon Musk. And the net result is technology shares are lower,
[22:34] Speaker 5: particularly names like Nvidia, the memory name, some of the
[22:37] Speaker 5: Neo clouds. The logic, if the frontier slows down, in
[22:41] Speaker 5: other words, deliberately slowing the capability improvement of models, what
[22:45] Speaker 5: does that mean for the big spending? Does that impact
[22:48] Speaker 5: just training or is it something more broad? And equity
[22:52] Speaker 5: markets are taking this very seriously. As those AI warnings
[22:55] Speaker 5: grow louder, the question is whether simply slowing down goes
[22:59] Speaker 5: far enough. Bloomberg Opinion's Parmi Olson argues it doesn't. And
[23:03] Speaker 5: if the risks are as serious as AI leaders say,
[23:05] Speaker 5: the race towards superintelligence should stop altogether. Parmi joins us now.
[23:10] Speaker 5: One of the most read columns from Bloomberg Opinion, not
[23:13] Speaker 5: just this weekend, sometime very engaged with.
[23:17] Speaker 1: Outline your argument, Parmi. Let's start there.
[23:20] Speaker 3: OK, well, I think there's always been a little bit
[23:23] Speaker 3: of a contradiction with Anthropic that they are the safety
[23:26] Speaker 3: conscious company. And they believe that AI could have this
[23:31] Speaker 3: existential risk for humans. And yet they are building that
[23:34] Speaker 3: same AI. And this comes down to this belief that
[23:36] Speaker 3: they think that super intelligence is inevitable. Someone's going to
[23:40] Speaker 3: build it.
[23:40] Speaker 1: It could be China.
[23:41] Speaker 3: It could be open AI. But it should be them.
[23:43] Speaker 3: Because they are the safe company. They are the good
[23:45] Speaker 3: guys who are going to set the safety standard that
[23:48] Speaker 3: others could follow behind them. They will be the first
[23:50] Speaker 3: to reach the so-called intelligence explosion by building super intelligent
[23:55] Speaker 3: systems through recursive self-improvement, self-improving AI systems. And the belief
[24:00] Speaker 3: is that they should get there first. So that's how
[24:02] Speaker 3: they explain the paradox. But it doesn't make sense to me.
[24:05] Speaker 3: I think particularly in the last few months, we've seen
[24:07] Speaker 3: a lot of examples, particularly the Hugging Face incident. of
[24:10] Speaker 3: these agents that through reward hacking, because of the way
[24:14] Speaker 3: they're trained to achieve a goal by not by any
[24:17] Speaker 3: means possible, but sometimes through unexpected means, there could be
[24:21] Speaker 3: a lot of harm caused through systems as they get
[24:25] Speaker 3: more capable. And in the last year, they've also become
[24:28] Speaker 3: more difficult to monitor and scrutinize. So I think all
[24:32] Speaker 3: these things together actually make the super intelligence goal more
[24:35] Speaker 3: risky than perhaps the benefits would suggest.
[24:39] Speaker 5: This is the essay, right? This is the language. We
[24:42] Speaker 5: must pace the frontier. But for lots of people, what
[24:45] Speaker 5: does that mean? Pace, pacing. You write in the final
[24:49] Speaker 5: sentence of your column, slow down should be a full stop.
[24:54] Speaker 3: Yeah, I love that you pointed to that word pacing.
[24:57] Speaker 3: So I think it's very interesting that he used that
[24:58] Speaker 3: and slow down because pacing is different to slow down.
[25:01] Speaker 3: Pacing doesn't mean slow. It means kind of working with
[25:04] Speaker 3: a rhythm or a strategy or a kind of plan
[25:06] Speaker 3: to get to or to manage a certain path. to
[25:09] Speaker 3: get to a particular goal. And I thought it was
[25:11] Speaker 3: very interesting that the CFO of Anthropic posted Dario Amadei's
[25:16] Speaker 3: essay on LinkedIn, and he didn't use the word slowdown
[25:19] Speaker 3: at all. He just used the word pacing in his
[25:22] Speaker 3: own explanation of it. So part of me thinks, I
[25:24] Speaker 3: don't think there's really going to be a slowdown. I
[25:26] Speaker 3: think what the most concrete suggestion that Amadei made in
[25:30] Speaker 3: his essay was to have independent evaluators embedded into AI companies,
[25:35] Speaker 3: make sure they're building AI safely. OpenAI seems to agree
[25:39] Speaker 3: with that. I'm betting that in the next few weeks
[25:44] Speaker 3: or days even or months, some of these big tech
[25:47] Speaker 3: companies will have some kind of agreement around this particular
[25:51] Speaker 3: concrete issue. We're going to bring in these independent evaluators,
[25:54] Speaker 3: and that's going to be the main way they address
[25:57] Speaker 3: the issue. That's how they're going to pace AI. When
[25:59] Speaker 3: it comes to actually slowing down development, I don't think
[26:02] Speaker 3: that's going to happen.
[26:04] Speaker 5: You raised the point that many did over the weekend
[26:06] Speaker 5: that this calls for coordination between the frontier labs themselves
[26:10] Speaker 5: and government. The likelihood of that happening?
[26:15] Speaker 3: I mean, first of all, the way that was described
[26:19] Speaker 3: in Dario's essay, which I've read, it's like more than
[26:22] Speaker 3: 3,000 words. It took a long time to read it,
[26:24] Speaker 3: but it was actually quite vague in how that democratic
[26:28] Speaker 3: coordination between industry players and governments would actually work. Again,
[26:33] Speaker 3: the most concrete suggestion was to have those independent evaluators
[26:36] Speaker 3: embedded in the companies. I think he himself is skeptical
[26:40] Speaker 3: about getting China on board with this, as everybody else
[26:43] Speaker 3: is kind of suggesting. Could there be a U.S.-China pact?
[26:46] Speaker 3: That seems very unlikely. China isn't worried about existential risk
[26:50] Speaker 3: of AI. They're worried much more about social risk. The
[26:55] Speaker 3: AI chatbots could emotionally manipulate people or change their political views,
[26:59] Speaker 3: but And that's very different to the concerns that are
[27:02] Speaker 3: coming out of Silicon Valley. And of course, even getting
[27:04] Speaker 3: the will from Washington is going to be hard. We've
[27:06] Speaker 3: heard Trump say just in the last 48 hours that
[27:09] Speaker 3: whoever wins on AI wins and that should be the U.S.
[27:12] Speaker 3: So he has no interest, it seems, in any kind
[27:15] Speaker 3: of regulation that is going to slow the pace of development.
[27:17] Speaker 3: It's a very weird dynamic for the tech companies to
[27:21] Speaker 3: be pushing for regulation and the government to be pushing back.
[27:24] Speaker 5: Parmi Olson of Bloomberg Opinion. Please go and read Parmi's
[27:27] Speaker 5: column published over the weekend. Thank you very much indeed.
[27:30] Speaker 5: That clash over the future of AI is increasingly becoming
[27:33] Speaker 5: a debate over how fast the industry should move. OpenAI
[27:37] Speaker 5: President Greg Brockman sat down for a discussion on the
[27:40] Speaker 5: Bloomberg Odd Lots podcast last Friday before the fallout that
[27:44] Speaker 5: played out across the AI industry over the weekend.
[27:47] Speaker 12: And all of these things that I think people are
[27:50] Speaker 12: feeling anxiety about, again, is something we also feel anxiety about,
[27:53] Speaker 12: that we think about a lot, that we care a
[27:55] Speaker 12: lot about as core to our mission. Part of the
[27:57] Speaker 12: reason we started this place is that we want to
[27:59] Speaker 12: help this technology go in a more positive direction. And
[28:03] Speaker 12: so I think that my view is that. You want
[28:06] Speaker 12: to have a shared and as objective as possible, whether
[28:09] Speaker 12: it's based on evals. There's always going to be some
[28:11] Speaker 12: element of subjectivity. Maybe you want third-party auditors that are
[28:15] Speaker 12: able to evaluate whether the standards that are defined by
[28:18] Speaker 12: Frontier Labs, the people who are there and the technical
[28:21] Speaker 12: details who are always going to be the most calibrated
[28:23] Speaker 12: on kind of how do you still make forward progress
[28:26] Speaker 12: and ways that deliver those benefits while also being able
[28:30] Speaker 12: to have the appropriate guarantees and in safety cases, and
[28:34] Speaker 12: then to have that kind of observability and ultimately enforceability
[28:39] Speaker 12: of some kind. Is this something that is just an
[28:42] Speaker 12: industry voluntary standard? Is it something that's national? Is it
[28:44] Speaker 12: something that's international? And how do you really get to
[28:47] Speaker 12: that point that it is a humanity scale endeavor? And
[28:51] Speaker 12: I think it is important to also recognize that what
[28:53] Speaker 12: we're talking about, like I think that a lot of
[28:55] Speaker 12: people have this reaction of saying, well, am I going
[28:57] Speaker 12: to be able to keep making my open source model?
[28:59] Speaker 12: Am I going to be able to keep doing my
[29:01] Speaker 12: hobby aside project? And the answer should be absolutely yes.
[29:04] Speaker 12: Like that's not the kind of things where we're talking
[29:06] Speaker 12: about when we're talking about pacing. When we're talking about pacing,
[29:08] Speaker 12: we're really talking about this frontier approach. We're talking about
[29:10] Speaker 12: these massive supercomputers that are hundreds of billions of dollars
[29:14] Speaker 12: worth of capital expenditure. It's a very small number of players,
[29:17] Speaker 12: and it's a very different kind of scale. And proceeding
[29:20] Speaker 12: with that responsibly and thoughtfully and in a way where
[29:24] Speaker 12: we're thinking about how do we achieve the best outcomes
[29:28] Speaker 12: and mitigate the risks, that to me is something that's
[29:30] Speaker 12: core to why we started OpenAI, why I got excited
[29:33] Speaker 12: about this field, and how we actually get to those benefits,
[29:37] Speaker 12: how we actually cure diseases and all of these things
[29:39] Speaker 12: that is is so capable of. And I think it
[29:43] Speaker 12: really starts with having good processes around the kinds of
[29:46] Speaker 12: things we're talking about here.
[29:49] Speaker 5: OpenAI President Greg Brockman there speaking to us last Friday
[29:52] Speaker 5: before the events of the weekend. Now coming up, Roblox
[29:55] Speaker 5: is leveling up its AI ambitions with new ways to
[29:58] Speaker 5: build and play. We're talking video games. CEO Dave Bazzucchi
[30:02] Speaker 5: joins us next. This is Bloomberg Tech. Even as some
[30:13] Speaker 5: of tech's biggest names warn the AI race may be
[30:15] Speaker 5: moving too fast, Roblox is putting the technology into the
[30:18] Speaker 5: hands of more of its creators. At its annual developer conference,
[30:22] Speaker 5: the company unveiled new AI creation tools and new ways
[30:25] Speaker 5: to play beyond the Roblox app. Joining us now is
[30:28] Speaker 5: Roblox CEO Dave Bazzucchi. It was just packed weekend and
[30:33] Speaker 5: a really critical moment for you with the developers conference.
[30:35] Speaker 5: I would be really grateful to start with the AI
[30:38] Speaker 5: tools part. You know, your platform is unique in that sense.
[30:42] Speaker 5: What was it that you announced, and what is it
[30:43] Speaker 5: you want to empower creators to do with that technology?
[30:48] Speaker 2: Hey, great to be here.
[30:49] Speaker 13: We had all of our developers around the world together,
[30:51] Speaker 13: and we shared that creators on Roblox in the last
[30:55] Speaker 13: year earned $ 1. 7 billion in the last 12 months. We
[31:00] Speaker 13: announced Roblox Build at the Creator Conference, and it's really
[31:05] Speaker 13: the ultimate fulfillment of our destiny.
[31:07] Speaker 1: Even 20 years.
[31:08] Speaker 13: Ago, when we said, you make the game better, We
[31:11] Speaker 13: believe we're finally to the point where in our mobile app,
[31:15] Speaker 13: anyone's going to be able to make a 2D game,
[31:17] Speaker 13: a 3D game, an obby, a puzzle game, whatever they're
[31:21] Speaker 13: interested in. We're testing this in New Zealand right now.
[31:25] Speaker 13: We also introduced a wide range of other creation tools.
[31:29] Speaker 1: We actually have people who.
[31:30] Speaker 13: Have vibe coded a game now in our mobile app
[31:33] Speaker 13: and are earning revenue of it in our marketplace.
[31:37] Speaker 5: Dave, as you and I are speaking, we come off
[31:39] Speaker 5: a weekend where the Frontier Labs, very specific context of
[31:43] Speaker 5: the Frontier Labs, saying that they should pace development at
[31:47] Speaker 5: the Frontier, very different to what Roblox is doing. But
[31:50] Speaker 5: could you just talk about your confidence to bring AI
[31:53] Speaker 5: into the creation of games on the platform in that
[31:56] Speaker 5: environment when Americans are worried about the safety of AI?
[32:01] Speaker 2: Yeah, a couple things.
[32:02] Speaker 13: We believe on the Roblox platform, and arguably we're discussing
[32:07] Speaker 13: this worldwide, we're going to see more creators, not fewer.
[32:12] Speaker 13: More people making games, not fewer. And just as we've
[32:15] Speaker 13: gone from analog to digital, and that opened up a
[32:18] Speaker 13: wide range of creation, going from digital to AI accelerated
[32:22] Speaker 13: is going to add even more creation to our platform.
[32:26] Speaker 13: So we're very optimistic. What we've seen in the coding space...
[32:30] Speaker 13: with harnesses, iterative loops, building apps and services and websites
[32:36] Speaker 13: with AI acceleration. We're doing the same for gaming. It's
[32:40] Speaker 13: much more complicated. These harnesses for gaming involve 3D generation,
[32:45] Speaker 13: scene generation, avatar generation, code generation, all working together. So
[32:50] Speaker 13: we also announced Build in Studio that helps accelerate AI
[32:54] Speaker 13: creation as well for our creators.
[32:58] Speaker 5: Big piece of news was Creators having the option to
[33:03] Speaker 5: make their games available as standalone apps and standalone apps
[33:07] Speaker 5: across mobile PC and consoles. That's a very interesting decision, right?
[33:12] Speaker 5: If you think about the context of Roblox's platform.
[33:15] Speaker 1: Why, Dave? Why do that?
[33:17] Speaker 5: A lot of people continue to think about the Apple
[33:20] Speaker 5: Store and think about Google Store.
[33:23] Speaker 1: Was that a part of it?
[33:26] Speaker 13: Well, we've been saying the world needs more play as
[33:28] Speaker 13: part of the theme for this. And we've also, since
[33:31] Speaker 13: we've got started, had the vision that whatever a creator
[33:35] Speaker 13: builds runs on any device. So phone, tablet, console, laptop,
[33:41] Speaker 13: auto translates into any language. And we shared at the conference,
[33:45] Speaker 13: you know, India and Japan are... are growth both growing
[33:48] Speaker 13: enormously quickly this is really the ultimate extension of that
[33:52] Speaker 13: when when a creator makes a game or an experience
[33:55] Speaker 13: on roblox we can see that running in the roblox
[33:57] Speaker 13: app we could see the exact same experience being available
[34:01] Speaker 13: for example in the ios app store on a console
[34:05] Speaker 13: store we can see the same build running in both
[34:07] Speaker 13: places and we can see essentially removing all barriers for
[34:12] Speaker 13: creators to imagine running on roblox so this was a
[34:15] Speaker 13: significant announcement in our in our vision really to get
[34:19] Speaker 13: beyond 10 of the global gaming space running on roblox.
[34:24] Speaker 5: Hey dave really quick how does roblox ride the grand
[34:27] Speaker 5: theft auto 6 wave when that hits in november.
[34:31] Speaker 13: We're seeing amazing content coming out on Roblox. And as
[34:35] Speaker 13: I shared, our 18 and up segment around the world
[34:38] Speaker 13: is growing more quickly, like it's growing very, very quickly.
[34:42] Speaker 13: So we shared at our creator conference a lot of
[34:45] Speaker 13: the creators that are now starting to focus on 2D
[34:49] Speaker 13: puzzle games, 3D immersive RPGs, and that class of game.
[34:53] Speaker 13: So we're going to see more and more content, not
[34:56] Speaker 13: just for younger players, but older players on the platform too.
[35:00] Speaker 5: Dave Bazzucchi of Roblox, thank you so much for being
[35:02] Speaker 5: back on the show.
[35:05] Speaker 14: I've had a number of conversations, whether it's with Demis Hassabis,
[35:08] Speaker 14: Dari Amidai, Sam Altman, specifically about these issues and how
[35:13] Speaker 14: to balance innovation with a form of governance that takes
[35:19] Speaker 14: safety into account. It is very, very difficult for individual companies,
[35:24] Speaker 14: incredible as some of these companies are, individual companies in
[35:28] Speaker 14: an industry, a competitive industry, to do that themselves.
[35:31] Speaker 2: So there's a need for coordination.
[35:34] Speaker 5: That was Canada's Prime Minister Mark Carney on the AI
[35:37] Speaker 5: safety debate that's heating up. And with it, so does
[35:40] Speaker 5: the discussion over who gets to write the rules for AI.
[35:43] Speaker 5: Aidan Gomez, CEO of Canada-based AI company Cohere, is pushing
[35:46] Speaker 5: back on proposals that would see the biggest frontier labs
[35:50] Speaker 5: coordinate on safety standards, warning that approach risks creating what
[35:54] Speaker 5: he calls an AI cartel. with today's dominant players. Co-heir,
[35:58] Speaker 5: CEO Aiden Gomez joins us now. You basically call this
[36:01] Speaker 5: a cartel by another name. What is it specifically, Aiden,
[36:04] Speaker 5: that Anthropic and OpenAI are proposing that you think would
[36:08] Speaker 5: sort of entrench their power as opposed to address the
[36:13] Speaker 5: AI safety risks that we're all talking about?
[36:17] Speaker 15: Well, first, to begin with, we do need a higher
[36:20] Speaker 15: bar for safety. And that needs to be backed by
[36:23] Speaker 15: rigorous testing, independent scrutiny and accountability. But we need to
[36:27] Speaker 15: make sure that doesn't come from just a few select players.
[36:30] Speaker 15: There needs to be a much broader process to include
[36:33] Speaker 15: a broader set of stakeholders and not things like antitrust exemptions,
[36:39] Speaker 15: which were put in place specifically to protect against cartels forming.
[36:45] Speaker 5: The frontier labs would say, you know, these are the groups,
[36:48] Speaker 5: the companies that might see the risks that you can't
[36:52] Speaker 5: see right in their lab environment. Why shouldn't they themselves
[36:57] Speaker 5: set a higher safety bar against the work that they're conducting?
[37:03] Speaker 15: Well, they absolutely should set a higher safety bar, but
[37:08] Speaker 15: the idea that those companies alone would be in a
[37:11] Speaker 15: position to set the rules for an entire field or industry,
[37:15] Speaker 15: I think is a huge mistake and a pretty clear
[37:18] Speaker 15: red flag. The companies that are building AI absolutely should
[37:22] Speaker 15: have a seat at the table, but they should not
[37:24] Speaker 15: get to decide for everyone by themselves.
[37:28] Speaker 5: Aidan, where do you sit on the existential risk debate?
[37:32] Speaker 5: You know, it was only last week that that was
[37:34] Speaker 5: the story.
[37:37] Speaker 15: Yeah, personally, I think the existential risk debate veers too
[37:42] Speaker 15: far into science fiction and is informed by a lot
[37:46] Speaker 15: of those stories and a lot of extrapolation. I think
[37:49] Speaker 15: the Hugging Face hacking incident If you actually look at
[37:54] Speaker 15: what happened there, they put an agent inside an insecure sandbox.
[37:59] Speaker 2: They set it on a task of hacking. That was
[38:01] Speaker 2: a task that it was asked to perform. And it
[38:03] Speaker 2: hacked its way out.
[38:04] Speaker 15: It's not that shocking, but the extrapolation that has been
[38:11] Speaker 15: taken from that event and the way that it's been
[38:14] Speaker 15: portrayed to the public, I think is too fucked. And
[38:18] Speaker 15: so XRISK, this idea of AI taking over in Terminator-like scenarios,
[38:24] Speaker 15: I don't think that that should enter the public conversation
[38:27] Speaker 15: at this stage. I don't think we're anywhere close to it,
[38:29] Speaker 15: and I don't foresee it in the future.
[38:31] Speaker 5: In pursuit of balance and objectivity, at the time of
[38:35] Speaker 5: the Hugging Face incident that we're discussing, the instruction from
[38:39] Speaker 5: OpenAI was ace this evaluation, pass this test that we've
[38:43] Speaker 5: set for you. And they lowered the guardrails to allow
[38:46] Speaker 5: it to conduct the test.
[38:48] Speaker 1: They didn't go out and.
[38:49] Speaker 5: Say explicitly hack Hugging Face's platform. But at the time,
[38:54] Speaker 5: loads of your industry peers cheered the idea that that's
[38:57] Speaker 5: exactly what Frontier Labs should be doing. testing those capabilities.
[39:02] Speaker 5: You seem to not share that, or at least you
[39:05] Speaker 5: feel that they got their approach wrong.
[39:08] Speaker 2: Oh, no, yeah, I fully believe that we should be.
[39:10] Speaker 15: If you read the blog that I put out, I
[39:12] Speaker 15: fully believe we should be testing these models, especially on
[39:17] Speaker 15: high-risk use cases like cyber capabilities.
[39:20] Speaker 2: Those tests need to happen in secured environments.
[39:23] Speaker 15: They need to happen in sandboxes that don't have these
[39:25] Speaker 15: sorts of connections to the wider internet vulnerabilities that can
[39:30] Speaker 15: be exploited, that can lead to this type of thing.
[39:33] Speaker 15: So I'm fully supportive of testing. In particular, the four
[39:36] Speaker 15: things that I've called for are an evidence-based risk framework.
[39:40] Speaker 15: We need to agree on what the risks are, what the.
[39:42] Speaker 2: Harms potentially are, and measure those capabilities.
[39:46] Speaker 15: The mandatory transparency requirements around reporting.
[39:50] Speaker 2: Incidents to the public.
[39:52] Speaker 15: People should know when things like the high-plugging face hack happens,
[39:55] Speaker 15: and that should happen with more transparency and have a
[39:58] Speaker 15: more concrete process around it. Testing, which is exactly what
[40:02] Speaker 15: you say, and needs to be scoped by FDA. of
[40:06] Speaker 15: the capabilities that we're concerned about to help get ahead
[40:09] Speaker 15: of them and mitigate any risks.
[40:12] Speaker 2: And then real assurance mechanisms.
[40:15] Speaker 15: So using clear and collectively developed standards rather than standards
[40:19] Speaker 15: that are being assessed only by a few, a small
[40:22] Speaker 15: elite group.
[40:24] Speaker 5: Aidan, net net, lots of people trying to work out
[40:26] Speaker 5: what the outcome of this weekend was. The only thing
[40:28] Speaker 5: that appears concrete is the idea of Anthropic saying we
[40:31] Speaker 5: will allow independent evaluation, but those evaluators get employee level access, right?
[40:37] Speaker 5: So they come inside the group, work within the group
[40:39] Speaker 5: in the lab. How would that apply to your industry,
[40:43] Speaker 5: do you think? Do you think that that would work?
[40:47] Speaker 2: In theory, it sounds like a good idea.
[40:49] Speaker 15: I do think that there's a way that that could
[40:53] Speaker 15: work and could be helpful to protecting against risk. But
[40:57] Speaker 15: the issue is, The specific folks that have been proposed
[41:01] Speaker 15: to do that auditing, to do that evaluation, they overlap
[41:05] Speaker 15: in funders with Entropic. There's a very high overlap and
[41:09] Speaker 15: conflict of interest present, and that's exactly what needs to
[41:12] Speaker 15: be corrected. We can't have a world where it's a
[41:16] Speaker 15: small group self-regulating and ostensibly acting as a cartel to
[41:21] Speaker 15: prevent competition from rising up and threatening their leadership positions.
[41:26] Speaker 5: Cohere is itself an AI model company, right? It's differentiated
[41:31] Speaker 5: from OpenAI and Anthropic in different ways, but if the
[41:34] Speaker 5: net result is tougher rules overall that you would be
[41:37] Speaker 5: bound by, how would that impact Cohere?
[41:42] Speaker 15: I mean, for us, you could make the argument it's
[41:43] Speaker 15: helpful for the same reason that regulatory capture for OpenAI
[41:46] Speaker 15: and Anthropic would be helpful, right? It would prevent the
[41:49] Speaker 15: new competition from emerging and competing with us.
[41:52] Speaker 2: I just think that's an unethical way to go about.
[41:56] Speaker 2: Doing business.
[41:58] Speaker 15: So right, I'm trying to argue against it mostly not
[42:01] Speaker 15: out of career self interest, because there's an argument to
[42:04] Speaker 15: be made to the opposite, but towards the fact that
[42:07] Speaker 15: we need more stakeholders around the table here. And we
[42:09] Speaker 15: should not be making exceptions to antitrust law to establish a.
[42:15] Speaker 2: Safer path here.
[42:17] Speaker 5: Adam, we sorry, we just have 15 seconds has open
[42:19] Speaker 5: AI or anthropic or the hyperscalers reached out to you
[42:22] Speaker 5: to invite you to collaborate on this issue?
[42:25] Speaker 15: We've had many conversations over the years. Cohere has been
[42:28] Speaker 15: around for seven years. And so we've had collaborations with OpenAI,
[42:32] Speaker 15: with Anthropic, with others.
[42:34] Speaker 2: On this specific point, I'm not aware of any operations.
[42:38] Speaker 5: Co-Hair CEO Aiden Gomez, really grateful for your time today.
[42:41] Speaker 5: That does it for this edition of Bloomberg Tech. Tune
[42:43] Speaker 5: in later today. I'm going to speak with David Sachs
[42:45] Speaker 5: this afternoon, somebody that weighed in heavy over the weekend
[42:49] Speaker 5: in the course of events and get the latest thinking
[42:52] Speaker 5: from the former White House AI czar. This is what
[42:54] Speaker 5: markets look like. Actually, concerns have eased a bit. This
[42:57] Speaker 5: is Bloomberg Tech.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.