# Data Engineering Podcast

A podcast by Tobias Macey. This show goes behind the scenes for the tools, techniques, and difficulties associated with the discipline of data engineering. Databases, workflows, automation, and data manipulation are just some of the topics that you will find here.

Feed: https://serve.podhome.fm/rss/1c0357c0-6aba-5766-a2d5-2090d8dab6bc

## Latest episodes

- [How Amazon Uses Telemetry and AI to Automate Physical Logistics](https://invisiblenetwork.ai/podcast/1193040557/062c65c0eef0) (2026-10-20, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6392805184014366512ed76dc5-e5ea-4887-8fa3-dc30d65c5d0b.mp3
- [Why Metadata Still Matters: From Data Catalogs to AI-Ready Semantic Hubs](https://invisiblenetwork.ai/podcast/1193040557/dace3009fba4) (2026-10-07, 85 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639269301622202034b471efd0-5c78-470d-b430-f1db25595736.mp3
- [Reducing Data Debt with Agile Ledger Architecture](https://invisiblenetwork.ai/podcast/1193040557/31c7fb808730) (2026-09-24, 50 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6392587911197837527d435cc5-ef2a-4406-8fd5-f8f5e9ea383e.mp3
- [What Context Really Means in Data Engineering and AI](https://invisiblenetwork.ai/podcast/1193040557/833cfa9e217b) (2026-09-15, 53 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6392442136782912425808093c-c664-4986-9964-121f5e14a01a.mp3
- [Specialized AI for Data Engineers: Inside Astronomer’s Otto](https://invisiblenetwork.ai/podcast/1193040557/769ded5f267e) (2026-08-27, 46 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639234304916782127a0815158-0341-48e3-ae7f-db841208a246.mp3
- [Why Multi-Agent Systems Need Shared State, Graph Semantics, and Governance](https://invisiblenetwork.ai/podcast/1193040557/9f1c9bc952c6) (2026-08-02, 63 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6392130619121105521c1e7e44-b4c7-4b7f-931a-c3e4a96937bf.mp3
- [Building the Context Flywheel for AI Data Agents](https://invisiblenetwork.ai/podcast/1193040557/285eacff0aba) (2026-07-06, 61 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6391889630903766738d7df31f-c5f6-48b1-8144-c4225fecd8aav1.mp3
- [Holding Kafka Right: Product-Friendly Streaming with TypeStream](https://invisiblenetwork.ai/podcast/1193040557/43749ab7e1a8) (2026-06-18, 50 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6391733588741892878ecda746-bff7-4c7e-b384-3fc00c01f44av1.mp3
- [Text to Data Products: Kaarvi’s End-to-End AI for Ingestion, Quality, and Dashboards](https://invisiblenetwork.ai/podcast/1193040557/c592fa19e85b) (2026-06-08, 53 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6391647681280765633f2d2645-06b3-49a6-90f2-3722747b3664.mp3
- [Scaling Graph Analytics Without ETL: Inside PuppyGraph’s Architecture](https://invisiblenetwork.ai/podcast/1193040557/6057ccb0e002) (2026-06-01, 55 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6391586951131897518fbec695-0bb3-4a0e-8ebd-eec46b1f65cfv1.mp3
- [Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and Kubernetes](https://invisiblenetwork.ai/podcast/1193040557/37e14d55f075) (2026-05-06, 59 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639137043323859577c83b710d-5a1a-4de0-bdf4-b6b264d0356bv1.mp3
- [The AI-First Data Engineer: 10–50x Productivity and What Changes Next](https://invisiblenetwork.ai/podcast/1193040557/55730aa5e142) (2026-04-07, 60 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6391120086482594914ba7683f-37a0-4b0d-b566-d901bf5b1b8dv1.mp3
- [Treat Metering Like Finance: Building Data Platforms for Consumption Economics](https://invisiblenetwork.ai/podcast/1193040557/5b1454ea6c1e) (2026-03-29, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639104201566165424af5f6f1d-ac60-45be-8e7d-da0d21335f1f.mp3
- [Beyond the PDF: Rowan Cockett on Reproducible, Composable Science](https://invisiblenetwork.ai/podcast/1193040557/37205c5288a4) (2026-03-22, 43 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390980655121795181d9b156f-8508-483e-9015-4b41c9a448ec.mp3
- [Beyond Prompts: Practical Paths to Self‑Improving AI](https://invisiblenetwork.ai/podcast/1193040557/f1d626c17ed2) (2026-03-16, 62 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639092222286896116ea4fb885-653c-45df-bfbd-3e9a171a99b6.mp3
- [Orion at Gravity: Trustworthy AI Analysts for the Enterprise](https://invisiblenetwork.ai/podcast/1193040557/4ce519337d46) (2026-03-08, 66 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390860976111881687bb6b052-3f3e-42a6-93d8-f50f5538cc8c.mp3
- [From Models to Momentum: Uniting Architects and Engineers with ER/Studio](https://invisiblenetwork.ai/podcast/1193040557/68f39b2758c2) (2026-03-02, 46 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639080061815379153a2b05b3f-d2ff-41d5-b4ed-a225bcb14685.mp3
- [From Data Models to Mind Models: Designing AI Memory at Scale](https://invisiblenetwork.ai/podcast/1193040557/2a8210dcd690) (2026-02-22, 58 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639073981323893202146bcc19-cf93-4fa2-9e96-70bb5d041afc.mp3
- [Prompt Management, Tracing, and Evals: The New Table Stakes for GenAI Ops](https://invisiblenetwork.ai/podcast/1193040557/08ae6acb1385) (2026-02-15, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63906715056530017229e94ae4-20eb-474e-a235-2d30233e840c.mp3
- [From Legacy to AI-Ready: How MongoDB AMP Accelerates Modernization](https://invisiblenetwork.ai/podcast/1193040557/e1a0ad5c3384) (2026-02-08, 47 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639061781047076086c6530cd9-89a7-4a9d-ab46-8c69a900125b.mp3
- [Branches, Diffs, and SQL: How Dolt Powers Agentic Workflows](https://invisiblenetwork.ai/podcast/1193040557/357df119b8bb) (2026-02-01, 57 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639055858748539200f2ffcea3-c0d7-4aeb-808a-4c6cf03db4bc.mp3
- [Logical First, Physical Second: A Pragmatic Path to Trusted Data](https://invisiblenetwork.ai/podcast/1193040557/e481b0ab4b1a) (2026-01-25, 41 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63904974303706807310097acb-6923-40ae-ab27-b43f45e4262e.mp3
- [Your Data, Your Lake: How Observe Uses Iceberg and Streaming ETL for Observability](https://invisiblenetwork.ai/podcast/1193040557/e6673af5afe3) (2026-01-18, 73 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639043759793013763a79c0604-4ef4-41b2-8eb6-a71de98d8b37.mp3
- [Semantic Operators Meet Dataframes: Building Context for Agents with FENIC](https://invisiblenetwork.ai/podcast/1193040557/782bdad8fbd8) (2026-01-12, 57 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639037763860713083c128628e-1237-42e4-8f78-ebf5250d0f51.mp3
- [Beyond Dashboards: How Data Teams Earn a Seat at the Table](https://invisiblenetwork.ai/podcast/1193040557/481bf06cc926) (2026-01-05, 50 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/639031572965788792b3f14090-e4cc-4db5-9a6a-b9e54d56eaa4.mp3
- [Unfreezing The Data Lake: The Future-Proof File Format](https://invisiblenetwork.ai/podcast/1193040557/6a5aed388cfe) (2025-12-29, 60 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390256322887438075afea1e6-dbe3-4081-8e85-dbbfcf0904d6.mp3
- [From Context to Semantics: How Metadata Powers Agentic AI](https://invisiblenetwork.ai/podcast/1193040557/77d75aa120b7) (2025-12-21, 67 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390193297761297808736db69-3731-4cea-a918-08b578b47c08.mp3
- [From Data Engineering to AI Engineering: Where the Lines Blur](https://invisiblenetwork.ai/podcast/1193040557/d1e53e0f707d) (2025-12-14, 27 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390134251626350147679eec8-3dfd-46e9-ba46-5eaffae40d45.mp3
- [Malloy: Hierarchical Data, Semantic Models, and the Future of Analytics](https://invisiblenetwork.ai/podcast/1193040557/91d788e1267e) (2025-12-08, 59 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6390075077084996601a990f24-c158-4040-82de-362219334ea5.mp3
- [Blurring Lines: Data, AI, and the New Playbook for Team Velocity](https://invisiblenetwork.ai/podcast/1193040557/f2ee44583e7c) (2025-11-24, 61 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63899541790653808255eadfd0-ff39-4fb0-92b1-76105ae0d98b.mp3
- [State, Scale, and Signals: Rethinking Orchestration with Durable Execution](https://invisiblenetwork.ai/podcast/1193040557/290596c2e816) (2025-11-16, 52 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638989316602517257596b3f26-c832-4345-9c4c-3eac26f91c59.mp3
- [The AI Data Paradox: High Trust in Models, Low Trust in Data](https://invisiblenetwork.ai/podcast/1193040557/0063280ced28) (2025-11-09, 52 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638983160301766546433fb150-92e7-42a5-a006-aacb4f6fee76.mp3
- [Bridging the AI–Data Gap: Collect, Curate, Serve](https://invisiblenetwork.ai/podcast/1193040557/653b53aca465) (2025-11-02, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389770810242681066b292405-3006-49d2-930a-cafa13f672ed.mp3
- [Beyond the Perimeter: Practical Patterns for Fine‑Grained Data Access](https://invisiblenetwork.ai/podcast/1193040557/5d37c089355f) (2025-10-27, 65 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638971245118030966f5967595-68b3-4325-b539-ecc104db97a8.mp3
- [The True Costs of Legacy Systems: Technical Debt, Risk, and Exit Strategies](https://invisiblenetwork.ai/podcast/1193040557/5f066ca77647) (2025-10-18, 65 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63896423394146651583df5a1e-9e9d-4420-81e6-37b9b676b765.mp3
- [Context Engineering as a Discipline: Building Governed AI Analytics](https://invisiblenetwork.ai/podcast/1193040557/09effb10ba57) (2025-10-11, 52 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63895812329159693841ea11a4-85ab-41f8-a111-1f187a4847df.mp3
- [The Data Model That Captures Your Business: Metric Trees Explained](https://invisiblenetwork.ai/podcast/1193040557/943a9ab80ab4) (2025-10-05, 62 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389530452840061764378b68e-257b-4de0-b0c4-411f64cfba45.mp3
- [From GPUs-as-a-Service to Workloads-as-a-Service: Flex AI’s Path to High-Utilization AI Infra](https://invisiblenetwork.ai/podcast/1193040557/aff009265e12) (2025-09-28, 57 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389469867830668118b206a3b-f37b-4dee-b6fd-79cf41d78e4b.mp3
- [From RAG to Relational: How Agentic Patterns Are Reshaping Data Architecture](https://invisiblenetwork.ai/podcast/1193040557/83ccedd25ebb) (2025-09-18, 53 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389375076436798105c0d58d0-9d30-444a-8029-39496dc19526.mp3
- [Duck Lake: Simplifying the Lakehouse Ecosystem](https://invisiblenetwork.ai/podcast/1193040557/3b9b30ea683e) (2025-09-10, 71 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638930608773775371e9f64793-a59e-445e-8cb4-6d23ceea6504v1.mp3
- [Aligning Business and Data: The Essential Role of Data Modeling](https://invisiblenetwork.ai/podcast/1193040557/83a533f4df2c) (2025-09-01, 67 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638923628895134615bb6d0710-dae8-492a-b479-78e1535ba495v1.mp3
- [From Academia to Industry: Bridging Data Engineering Challenges](https://invisiblenetwork.ai/podcast/1193040557/28820d31d7e5) (2025-08-26, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389176645945800329db6232b-703f-4282-ac92-0bb4b75864c5v1.mp3
- [High Performance And Low Overhead Graphs With KuzuDB](https://invisiblenetwork.ai/podcast/1193040557/702c972dd673) (2025-08-18, 62 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6389114770602141280ad09eb1-5e84-4cb6-8a9c-b9ce106d7361v1.mp3
- [Bridging Data and Decision-Making: AI's Role in Modern Analytics](https://invisiblenetwork.ai/podcast/1193040557/1858c480a327) (2025-08-12, 71 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63890555012731909146f77e53-87a3-4d34-8f85-478d79032381v1.mp3
- [From Bits to Tables: The Evolution of S3 Storage](https://invisiblenetwork.ai/podcast/1193040557/e33f72a4a1d1) (2025-08-05, 51 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/63889948293275633886f7b49d-9fff-4a3a-a61d-b11438ec0987v1.mp3
- [Revolutionizing Python Notebooks with Marimo](https://invisiblenetwork.ai/podcast/1193040557/9068910e41cd) (2025-07-28, 52 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6388924891606952449d69d6aa-46a0-442c-89d4-3cbca0c3d4e6v1.mp3
- [Warehouse Native Incremental Data Processing With Dynamic Tables And Delayed View Semantics](https://invisiblenetwork.ai/podcast/1193040557/1dc2308d0cb0) (2025-07-21, 56 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6388865463338603060e75e49c-19da-4d9d-8b28-85d634fc9126v1.mp3
- [Streamlining Data Pipelines with MCP Servers and Vector Engines](https://invisiblenetwork.ai/podcast/1193040557/c31536d0a826) (2025-07-15, 53 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638881398624809645ebadfc9e-3b88-4432-9a38-d8b87f84e35av1.mp3
- [Foundational Data Engineering At Two Sigma](https://invisiblenetwork.ai/podcast/1193040557/88093c218edb) (2025-07-06, 56 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/638874400453112501f4a1f4fa-2dc1-4d97-b5c7-6c43ab854a68v1.mp3
- [Enabling Agents In The Enterprise With A Platform Approach](https://invisiblenetwork.ai/podcast/1193040557/86069f4354fa) (2025-06-29, 55 min): audio https://op3.dev/e/dts.podtrac.com/redirect.mp3/serve.podhome.fm/episode/f6ff0caa-931b-4c08-bfdd-08dc7f5cd336/6388683154866773368ee4f2b1-e648-4cc5-ad6e-810049cb0b98v1.mp3

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them.