Podcasts · The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Imagine while Reasoning in Space: Multimodal Visualization-of-Thought with Chengzu Li - #722

· 42 min · Episode 722

Episode Imagine while Reasoning in Space: Multimodal Visualization-of-Thought with Chengzu Li - #722

0:00 42:11

Today, we're joined by Chengzu Li, PhD student at the University of Cambridge to discuss his recent paper, “Imagine while Reasoning in Space: Multimodal Visualization-of-Thought.” We explore the motivations behind MVoT, its connection to prior work like TopViewRS, and its relation to cognitive science principles such as dual coding theory. We dig into the MVoT framework along with its various task environments—maze, mini-behavior, and frozen lake. We explore token discrepancy loss, a technique designed to align language and visual embeddings, ensuring accurate and meaningful visual representations. Additionally, we cover the data collection and training process, reasoning over relative spatial relations between different entities, and dynamic spatial reasoning. Lastly, Chengzu shares insights from experiments with MVoT, focusing on the lessons learned and the potential for applying these models in real-world scenarios like robotics and architectural design.

The complete show notes for this episode can be found at https://twimlai.com/go/722.

The episode on the publisher’s site (opens in a new tab)

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them. Explicit shows are for adult profiles only, and children’s profiles hear only children’s shows. More about this list.