Podcasts · DGT podcast: Languages and Technology
Episode 14: Can AI treat all languages equally? Students put LLMs to the test
· 29 min · S2 E14
Episode Episode 14: Can AI treat all languages equally? Students put LLMs to the test
How can we make sure AI works fairly across Europe’s languages?
In this episode of DGT podcast: Languages & Technology, Anna Holmén speaks with Pilar Sánchez Gijón from the Autonomous University of Barcelona and Susana Valdez from Leiden University about the EU MMLU project.
In this project, European Master’s in Translation students are translating an AI evaluation dataset into the EU official languages. They are gaining practical experience while helping develop a multilingual benchmark for large language models.
We also hear from DG Translation's Anna Kokkinidou, on how DG Translation and the European Master’s in Translation network collaborate on this project and more.
The guests discuss the challenges of adapting AI evaluations to different languages and cultures, the importance of human expertise, and plans to expand the benchmark to all 24 EU official languages.
More on the EU MMLU and the EU institutional LLM: An EU Institutional LLM to prevent digital language extinction - Multilingualism, translation and language-based AI services
Links in the notes 1
Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them. Explicit shows are for adult profiles only, and children’s profiles hear only children’s shows. More about this list.