Duck Tales: The role of user research, A/B testing, and "dogfooding" at DuckDuckGo (Ep.31)

20 May 2026 · 19 min

Episode Duck Tales: The role of user research, A/B testing, and "dogfooding" at DuckDuckGo (Ep.31)

0:00 19:20

In this episode, Beah (Chief Product Officer) and Zac (SVP, Insights) discuss our approach to user research, how we run robust research while respecting user privacy, and why moving quickly matters.

Disclaimers: (1) The audio, video (above), and transcript (below) are unedited and may contain minor inaccuracies or transcription errors. (2) This website is operated by Substack. This is their privacy policy.

Beah: Hello and welcome to Duck Tales, where we go behind the scenes at DuckDuckGo and discuss the stories, technology, and people that help build privacy tools for everyone. In each episode, you’ll hear from employees about our vision, product updates, engineering, or approach to AI. In today’s episode, we’re going to talk about user research. And so my guest here is Zac. Zac, do you want to introduce yourself?

Zac: Yeah, hi, Zac Pappas. I am on the Insights team at DuckDuckGo. I’ve been here for coming up on 14 years and pretend to participate in a lot of market research, user research, and other conversations that we have where we try to infer what users want and whether or not we’re giving it to them.

Beah: Sweet, thanks. And I’m Beah. If you haven’t met me, I’m on the product team at DuckDuckGo. All right, let’s launch into it. I have questions for you, Zac. Are you ready?

Zac: Ready as you’ll ever be.

Beah: All right, so let’s start. Why don’t you give me an example? Like when we’re talking about validating things with user research, just like let’s set up an example of what that means and what that would look like.

Zac: I think so. Yeah. So I guess starting with what is user research and why do we do it? I think the obvious answer that people tend to think about are, you know, sitting in an interview or doing some sort of focus group and learning from people. But really, for us, user research is kind of all of that plus, you know, internal dialogue and real testing like A/B testing. So to put it, you know, simply, I guess, user research for us is just trying to be wrong very productively and being efficient at being wrong productively over time. So for us, everything really starts out as a guess or an assumption about users or their behavior about, you know, what is upcoming in the market and research checks that, you know, our assumptions are correct or at least as close to correct as possible. And we try to do that as cheaply as possible, meaning quickly without spending more time on it than what we’re gaining in terms of the product that we’re creating. So an example could be our homepage at DuckDuckGo.com because our homepage has been around for as long as I think DuckDuckGo has been around, but has gone through many, many, many forms. We’ve done tons of A/B testing on it, showing one version of it to some users and a different version to other users. We’ve done a lot of user testing on it, like sitting down with actual consumers of the homepage or users of the homepage. Everything from discussing with parents how they perceive the homepage. So a lot of subtlety that goes into what we think maybe appears as a pretty simple layout or design.

Beah: I had a little connection issue there for a second, but I think it’s alright. Let’s forge ahead. Sorry, audience. So, the homepage, I think, as you pointed out, sort of like deceptively, seems deceptively simple, it’s not highly functional software, it’s a page, there’s a few boxes on it, a field, but like what makes it so tricky and worthy of user research.

Zac: Yeah, good question. Because it is kind of subtle, or I guess it doesn’t seem very obvious. So if you think about it, the homepage is really where we have a big mix of different user types coming in. Making any assumption about a part of our product usually means that you have to infer who is using it, meaning are they younger or older? Do they have a lot of context about what they’re doing or very little? So our homepage is actually a mix of different user types. There’s a lot of returning users who…

Links in the notes 2

The episode on the publisher’s site (opens in a new tab)

Shows are listed from Apple Podcasts' charts and directory, and Podcast Index where it is used. Every show and episode belongs to its publisher and plays from the publisher's own feed; this site hosts none of them. Explicit shows are for adult profiles only, and children’s profiles hear only children’s shows. More about this list.