Practical workflow orchestration
· 58 min · Episode 291
Episode Practical workflow orchestration
Workflow orchestration has always been a pain for data scientists, but this is exacerbated in these AI hype days by agentic workflows executing arbitrary (not pre-defined) workflows with a variety of failure modes. Adam from Prefect joins us to talk through their open source Python library for orchestration and visibility into python-based pipelines. Along the way, he introduces us to things like Marvin, their AI engineering framework, and ControlFlow, their agent workflow system.
Sponsors:
• WorkOS – A platform that gives developers a set of building blocks for quickly adding enterprise-ready features to their application. Add Single Sign-On (Okta, Azure, Google, Microsoft OAuth), sync users from any SCIM directory, HRIS integration, audit trails (SIEM), free magic link sign-in. WorkOS is designed for developers and offers a single, elegant interface that abstracts dozens of enterprise integrations. Learn more and get started at WorkOS.com • Shopify – Sign up for a $1/month trial period at shopify.com/practicalai • Notion – Notion is a place where any team can write, plan, organize, and rediscover the joy of play. It’s a workspace designed not just for making progress, but getting inspired. Notion is for everyone — whether you’re a Fortune 500 company or freelance designer, starting a new startup or a student juggling classes and clubs.
Featuring:
• Adam Azzam – LinkedIn, X • Chris Benson – Website, GitHub, LinkedIn, X • Daniel Whitenack – Website, GitHub, X
Show Notes:
• Prefect • Marvin • ControlFlow
Upcoming Events:
• Register for upcoming webinars here!
Links in the notes 16
- https://workos.com
- https://workos.com/
- https://www.shopify.com/practicalai
- https://www.linkedin.com/in/adam-azzam
- https://x.com/AAAzzam
- https://chrisbenson.com
- https://github.com/chrisbenson
- https://www.linkedin.com/in/chrisbenson
- https://x.com/chrisbenson
- https://www.datadan.io/
- https://github.com/dwhitena
- https://x.com/dwhitena
- https://www.prefect.io/
- https://www.prefect.io/marvin
- https://www.prefect.io/controlflow
- https://practicalai.fm/webinars
Chapters
Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them. Explicit shows are for adult profiles only, and children’s profiles hear only children’s shows. More about this list.