🧠 The Understanders research: User perceptions of model selection in ChatGPT

People started talking about the model selection UI in ChatGPT last weekend, so we interviewed 51 paying users to understand how they feel about model switching

🎉 Welcome to the inaugural share-out of original primary research conducted by The Understanders. I’m not certain how often we’ll be able to do this, but this first foray has been a really fun endeavor. Many thanks to the kind folks at Outset who provided complimentary use of their research tool and who paid for the study’s participants.

---

There’s been an interesting lag in the discourse about AI. We’ve become so intrigued by what it’s capable of and how quickly those capabilities are advancing that we haven’t had the time, focus, or inclination to examine the interfaces and products that we’re using to engage with it.

But it feels like something shifted just in the past week on that front. First we had Casey Newton at Platformer talking about DeepSeek’s user experience as a competitive differentiator and the implications for AI lab product teams. Then something of a meme popped up this past weekend about the model selection UI in ChatGPT. Here’s Benedict Evans’ post about it, but I saw others (including Dare Obasanjo and, again, Casey Newton) share similar screenshots and observations, too.

It’s an interesting question: do most users of ChatGPT feel this way? How common is model switching as a behavior, and how well do people understand it? It’s the perfect candidate for our first piece of original research.

In partnership with Outset, we recruited 51 English-speaking paid subscribers to ChatGPT in the US, UK, and Australia to participate in an AI-moderated, semi-structured interview about this exact topic. It was my first experience using Outset, and it was a pretty powerful one: the participants were all sourced and interviewed in under a day on the platform, and it produced a summary report in a matter of minutes. (I used the automated reporting to direct my own synthesis, but it was a huge help nonetheless.) All told, we completed the study in three days. (Note that any video clips were from participants who granted consent for their interviews to be shared without their names attached online.)

Research Questions

We wanted to know:

1. How aware are paid users of the ability to switch between models?

2. For what reasons do they switch models, if they ever do?

3. How confident do they feel in their ability to match their objective to the most appropriate model?

4. What are their overall perceptions of the ability to switch between models?

Here’s what we found, and— in a bit of unforgivable and shameless clickbait— the interesting paradox we uncovered:

Findings