# RAG Risks: Why Retrieval-Augmented LLMs are Not Safer with Sebastian Gehrmann - #732

An episode of [The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)](https://invisiblenetwork.ai/podcast/1116303051), 21 May 2025, 57 min.

Audio: https://pscrb.fm/rss/p/traffic.megaphone.fm/MLN9960265469.mp3?updated=1747852071

Today, we're joined by Sebastian Gehrmann, head of responsible AI in the Office of the CTO at Bloomberg, to discuss AI safety in retrieval-augmented generation (RAG) systems and generative AI in high-stakes domains like financial services. We explore how RAG, contrary to some expectations, can inadvertently degrade model safety. We cover examples of unsafe outputs that can emerge from these systems, different approaches to evaluating these safety risks, and the potential reasons behind this counterintuitive behavior. Shifting to the application of generative AI in financial services, Sebastian outlines a domain-specific safety taxonomy designed for the industry's unique needs. We also explore the critical role of governance and regulatory frameworks in addressing these concerns, the role of prompt engineering in bolstering safety, Bloomberg’s multi-layered mitigation strategies, and vital areas for further work in improving AI safety within specialized domains. The complete show notes for this episode can be found at https://twimlai.com/go/732.

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.