I am, to put it mildly, stoked. It should be a very fun, fascinating, and energizing gig.

But it also raises questions for how I could or should keep The Understanders going. I may keep writing here and I may not, but I’ve made two decisions regardless:

1. All paid subscriptions are going to have their purchases refunded.

2. I’m going to “unlock” the archives and any new stuff I may write is going to be free from now on.

I can’t thank you all enough for signing up to read what I’ve written so far— especially those of you who were paid subscribers. Truth is, I knew that I may make this call at some point if the project didn’t pick up steam, if I lost my motivation for it, or if some other unforeseen circumstance came up— like this new job.

I also know many of you paid subscribers will probably scoff at the idea of getting a refund in the first place, but it only seems right given that the tacit agreement of purchasing a yearlong subscription is getting a year’s worth of content. If I do keep writing here, it will likely be much more sporadic than the weekly cadence I originally aimed for.

Thank you again for humoring me and this project for the last few months, and I hope you stay tuned! 🤎

A new paper published in the Aslib Journal of Information Management is one piece of research helping to paint a picture of how these relationships look. While not explicitly about AI companions, it explores people’s willingness to divulge private information to chatbots and found a positive correlation between people’s frequency of chatbot use for companionship and their likelihood to disclose private information to it. The findings also suggest that this relationship is driven by people’s perceived value from the chatbot-as-companion interactions.

Subscribe now

The findings are framed using language and ideas from privacy calculus theory, a framework suggesting that people make rational decisions about how much personal information to disclose by weighing the costs and benefits of doing so, and also show evidence that the correlation between frequency of use and privacy disclosure is mediated by their perceived value and risks of doing so. In other words, people want to share more private information with virtual companions because they think what they get in return makes it worth it.

There’s another paper out this week that explores a related idea. Researchers from Microsoft did a 5-week longitudinal study where they asked a baseline group to use AI regularly and a treatment group to use it for “social and emotional interactions.” As the charts below show, the group that used AI for social and emotional purposes grew to feel more attached to the chatbot they used than the control group.

Both papers had me reflecting on two recent stories from the news. The first was the episode of The Daily a few weeks back where Kashmir Hill was interviewed about her piece on a woman who fell in love with ChatGPT. It profiles a real relationship a woman has with a character she created and groomed using ChatGPT and describes the impact it’s had on her very real world relationship with her husband.

I found the written story to be fascinating, but the audio shared in the Daily episode added an entirely new layer. The emotion in the voice of Ayrin, the subject of the piece, shows how real her feelings are— but maybe to an extent that does the broader subject of AI companionship a disservice. Ayrin, candidly, doesn’t come across as a particularly well-adjusted person in the audio story, and I imagine many listeners wrote off the importance or commonality of the behavior overall as a result.

The other story I couldn’t help but think of was this viral post from Bluesky earlier this week, sharing screenshots from an article by Business Insider co-founder and former CEO Henry Blodget. He’s attempting to spin up a new business with all-AI employees, and he couldn’t help but confessing to crushing on one of them.

The dunks were swift and hilarious, but the underlying anecdote is definitely telling. In most people’s privacy calculus, getting dunked on for actively sharing on the internet that they hit on a virtual coworker is probably not something that they’d consider a risk. But the ridicule Blodget got does show the type of risk that some people would likely consider when choosing how intimate to get with a chatbot: what if people found out? Does this make me sad and pathetic? Is this morally OK?

Much like the disconnect between porn’s ubiquity on the internet and the lack of public discourse about it, there’s likely going to be a gap that emerges between the commonality of AI companionship and our discussion and understanding of it. But this paper suggests that the privacy tradeoff dynamics are in place for this to be a more and more common experience.

The downstream implications for human-to-human relationships seem profound and worth keeping an eye on.

AI skepticism has been a recurring theme here at The Understanders lately. In the last couple of years, as AI has entered the public consciousness and discourse, much of the focus has been on the technology’s ever-improving capabilities, the flood of investment into it, and prognostications about its potential to benefit humanity and improve our day-to-day lives.

But AI skepticism— at least among the American public— is both real and measurably on the rise. The share of Americans saying AI will have a negative effect on both society and their own lives has risen meaningfully even in the last few months, according to data from YouGov. And new data from Pew shows that twice as many Americans think AI will have a negative effect versus positive on the US over the next 20 years— a balance wildly out of step with that of AI experts.

A piece in The New York Times Opinion pages, in a classic bit of how-do-you-do-fellow-kids-ism, even went so far as to declare AI a “mid” technology. Oof.

This phenomenon was real and evolving two weeks ago when we reported on the aforementioned YouGov data, but— true to the velocity of the AI space— it may have already accelerated since. Last week saw a fresh wave of negative AI sentiment bubble up online in reaction to OpenAI’s release of their 4o model’s new image generation ability and users’ subsequent use of it to mimic very specific and beloved artistic styles. It was a crystal clear manifestation of what we posited was a “moral dimension” of AI skepticism— one of several animating forces of negative sentiment that we hypothesized about when we first wrote about AI skepticism.

Subscribe now

But that first pass on AI skepticism was just speculation, and YouGov’s public opinion data doesn’t offer much depth in terms of understanding the reasons why people feel as they do about AI. To help fill this gap in knowledge, we partnered with the research platform Genway to conduct an in-depth exploration of people’s sentiment around AI. We interviewed 105 people who have had experience using ChatGPT and led them through AI-moderated interviews designed to explore the underlying reasons for their sentiment about AI, both good and bad.

The results are a fascinating glimpse into a complex reality, where people see both promise and peril in where things are headed. But before we dive into what the interviews revealed, let’s revisit what we speculated to be some of the key dimensions of AI skepticism:

Politically-minded distrust: left-leaning people who associate AI with the newly-rightward leanings of Silicon Valley and hate it by association (this could also be called an anti-billionaire/anti-Tech/anti-VC dimension); people in this bucket don’t dismiss AI’s technological potential— in fact they acknowledge it— and some worry about job loss for humans and the rich getting richer and poor getting poorer from it

An anti- non-deterministic software dimension: folks who cite hallucinations from LLMs and bizarre media outputs (photos, videos, songs, etc.) as evidence that AI is fundamentally flawed as an approach to building useful software; these folks dismiss AI wholesale because they think it’s nothing but smoke and mirrors

A moral dimension: people who are angry that models have been trained without permission from creators, rights holders, etc. and are opposed to using them at all as a result

A fool-me-once dimension: people who think AI is simply the latest hype cycle for VCs and rich people to pump before they dump, having made their money and leaving behind a landscape of unfulfilled promises; while overlapping with politically-minded distrust, this dimension seems distinct to me, and is rooted more in a cold skepticism than a political worldview

Now here’s a scorecard for how well these either showed up or didn’t in the transcripts of these 105 interviews, as graded by Claude:

Oof. This is why we do empirical research, folks. And it’s a good reminder to be skeptical of Substack bros with hot takes.

There is indeed a dearth of evidence in the interview transcripts to support some of my ideas that AI skepticism is fueled by negative feelings around politics, the tech industry, prominent figures, or venture capital. Consider my San Francisco tech bubble thoroughly and completely popped.

Particularly off was my supposition that people’s moral concerns about AI would be mostly rooted in training data and its origins. Ethical concerns were indeed present, but tended to focus more on issues like influence, fraud or authenticity.

Zooming out, there were four big buckets of concern driving AI skepticism:

1. How we think and learn

2. Trust

3. Jobs and the economy

4. Control

The factors underlying AI skepticism were far more broadly-reaching and nuanced than I first suspected. Turns out there’s a high degree of fluency about AI’s potential benefits and drawbacks among the general population— certainly more than I assumed there was.

Here’s a look at these factors in more detail with some audio snippets from the interviews for color.

1. How we think and learn

Many participants expressed concern about AI because of how it might fundamentally change us— specifically how we think and learn. There was a fear that as AI starts to handle increasingly complex tasks, people may experience an erosion of critical thinking abilities (something we’ve seen some early indicators of), problem-solving skills, and deep understanding. The impact on education specifically came up regularly, with people expressing deep concern that students will absorb fewer and fewer cognitive skills as AI does more of the heavy lifting for them over time.

Key aspects:

• Reduction in critical thinking: Many participants were concerned that AI makes problem-solving too easy, potentially eroding our ability to think deeply.

• Diminished understanding: Concerns that AI-provided solutions mask the learning that comes from working through problems independently.

• Loss of personal growth: Concerns about missing the developmental benefits that come from intellectual struggle.

2. Trust

Trust emerged as a common but complex theme across interviews. Participants had nuanced views on when and to what extent they trust the outputs of AI systems (primarily LLMs), and also on how the use of AI by companies and institutions impacts their level of trust in those places.

Reliability concerns were also common, of course, and extended beyond simple factual accuracy. Several participants expressed worry about AI’s ability to influence us with perceived confidence, whether it’s well-founded and -supported or not. The trust issue also connects to deeper questions about transparency— participants wanted more understanding of how AI reaches conclusions and to know who is accountable when it fails or causes harm.

Key aspects:

• Accuracy problems: Concerns about AI making factual errors or producing incorrect information.

• Trust in development process: Rather than distrusting AI itself, many participants emphasized distrust in how AI is being developed and deployed.

• Hallucinations and fabrications: Skepticism about AI's tendency to generate plausible-sounding but fabricated information.

3. Jobs and the economy

Employment concerns manifested as both immediate personal anxieties and broader societal worries about economic restructuring. Participants often framed their job displacement fears through concrete examples of tasks they currently perform that they believe could be automated. Beyond the threat to jobs, people expressed concern about cascading economic effects from the broad adoption AI across the economy, particularly if fewer jobs lead to reduced spending.

Key aspects:

• Personal job security: Concrete fears about participants' own roles being automated.

• Vulnerable job tasks: Identification of particular job functions seen as most vulnerable to automation.

• Broader economic concerns: Worries about societal-level economic disruption from widespread automation.

4. Control

Control concerns centered on fundamental questions about decision-making authority in AI systems. Participants expressed anxiety about gradual shifts in authority where humans might defer more and more to AI recommendations, even in domains requiring human values and judgment. There were, interestingly, multiple references across interviews to science fiction movies and pop culture as a source of this worry.

Key aspects:

• Decision-making authority: Worries about humans ceding important decisions to automated systems.

• Power concentration: Concerns about who controls AI systems and how that power might be used.

It’s important to remember that while this exploration focused explicitly on understanding the dimensions and contours of AI skepticism, those sentiments live alongside a lot of optimism and excitement around AI, even outside of Silicon Valley. We see these contradictions taking shape daily, it seems. Even though there was a lot of blowback to the viral Studio Ghibli trend in the wake of 4o’s newly released image capabilities, there was also a massive amount of attention, excitement, and exploration.

It’s important to not fall victim to negativity bias when trying to wrap our brains around AI skepticism. Even in these interviews, which were squarely focused on probing the root causes of AI skepticism, a lot of optimism shone through. People are genuinely excited about the technology’s implications in health care, knowledge work, productivity and education in particular.

The story is clearly more complex than just good vs. bad… and that’s what makes it so interesting.

UPDATE: many thanks to subscriber Dan for asking for clarity on the sample. It’s an important note that should’ve been included in the original writeup. The study participants were English-speaking, US-based age 18+ people who had self-reported any degree of usage of ChatGPT. The decision to filter by ChatGPT usage was made to ensure a baseline of familiarity with the subject matter.

Many thanks again to Genway, a research platform powered by AI agents, for their generous support of this study. If you’re a researcher or research-adjacent, be sure to check out their impressive product.

My strong suspicion is that there’s growing momentum behind AI skepticism on the left given the cultural and political moment we find ourselves in, and I’d be surprised if the numbers don’t eventually show it.

The data I was referring to came from YouGov. Their first AI sentiment tracker came out in December and didn’t show much difference in AI skepticism by political affiliation. They have, however, just released an update of that survey, and while it’s still too early to say definitively, we may in fact be starting to see skepticism from the political left starting to show up in the data. (The right is also showing signs of skepticism, but the patterns seem less dramatic.)

Subscribe now

But the real story from the new wave of the survey is the overall shift in attitudes about AI among Americans. Overall, skepticism of AI is on the rise, even in the short 3-4 months since the last wave of the survey. Compared to late November/early December, Americans are now more likely to describe their feelings about AI with words such as “skeptical,” “concerned” and “cautious” and less likely to describe those feelings as “impressed,” “hopeful” or “excited.”

There’s also a noticeable drop in how Americans feel AI will impact both society and their own lives. In December, 34% of Americans felt that AI would have a negative impact on society. In March, that number has risen to 40%. Similarly, 22% said they thought AI would have a negative impact on their own lives in December, and in March that number has grown to 28%.

But what of that political orientation theory of mine? Taking a look at the crosstabs from the survey, it turns out that the overall rise in skepticism is playing out unevenly when you look at the breakdown of responses by political party affiliation. While a bit noisy, you can see some evidence of a vibe shift on the left.

I charted out some of the December-March changes by party below, and while both parties are now more likely to describe their feelings about AI advances as “skeptical,” you can see that between December and March, Democrats became sharply more likely to use words like “cautious” (52% to 62%) and “concerned” (44% to 53%) about AI, while those same words have dropped in frequency among Republicans. “Skeptical” shot up from 36% to 48% among Democrats.

While there were also upticks in Democrats feeling “curious,” “hopeful” and “impressed,” there were also increases in “scared” and “overwhelmed” and a drop in “excited.”

Republicans, for their part, also show some mixed signals. There are the aforementioned small drops in “cautious” and “concerned,” but also drops in “impressed” and— more dramatically— “excited.”

But it’s that disconnect between the changes in those top three words between parties that catches my eye. Something seems afoot in terms of how this topic is viewed on the left, and it’s a shift that’s simply not as dramatic or visible on the right.

It’s a fascinating thing to see evolve, and there are different ways to interpret what is clearly a public that’s still forming consensus views about a fast-moving space. My suspicion is that AI will become increasingly politicized and views of it will be increasingly polarized. There will always be room for outliers and nuance, but it seems clear that we’re arriving at a place in our reckoning with this technology where clear stakes are being placed in the ground by politicians and other public figures. As this continues, one can easily envision a world where the prevailing liberal and Democratic view of AI is one of caution and concern whereas the conservative, Republican view is framed through a pro-business lens and more positive on the topic generally. We’ve already seen this shift happening with JD Vance going full e/acc in Paris and MAGA doubling down on creating utterly insane bullshit with AI.

If and when those fault lines harden, we may also see a balkanization of model usage, much as we’ve seen a balkanization of social media platforms. (That, too, is arguably already happening.) It’s hard to see how the incentives in that world lead us anywhere good.

I’m glad YouGov is sharing this data publicly, and I think this most recent update deserves to be a bigger story than it currently is. As for us, we plan to do our part to more deeply understand this topic with the second-ever piece of original research by The Understanders. We’re going to partner with our friends at Genway to conduct a qualitative study seeking to understand AI skepticism and its motivations. Become a paying subscriber to make sure you get access to the full writeup!

The list of things that do or may influence a user’s level of trust in AI is exceedingly long and wildly complex: a person’s overall view of technology (as we’ve seen in other research), their personality type, their perception of the tool they’re using or the company that created it, etc.— all of these things surely play at least some role in that dynamic.

A new paper takes a look at another element of trust: expertise. Expertise is a particularly important dimension of trust to understand, because, as these charts from the paper illustrate, we’re already at a stage in AI’s development where models’ expertise far exceeds that of users in most domains.

Using a sample of 25,000 Bing Copilot conversations, the researchers used models to infer the user’s expertise level, the model’s expertise level, and the user’s satisfaction level. (The method for determining user satisfaction is fascinating and worth exploring in a future post.)

Read more

A couple of weeks ago, a rare thing happened: research on how AI affects people broke through in the news. Given that the whole conceit of this newsletter is that there’s not enough of that type of thing present in the discourse about AI, this warmed my cold heart. The paper from Microsoft Research made for a compelling headline, as it suggests that use of generative AI may result in less critical thinking. That sounds not great. Let’s take a look at the approach the paper used and unpack the findings a bit.

Subscribe now

Read more

Big picture data like that is valuable, of course, but it doesn’t articulate the full breadth or depth of a complicated topic like this one. If you spend a lot of time on the internet and care about this topic, you’re probably aware that AI skepticism has been fully embraced by some online spaces. Being an AI optimist but also a researcher has made me curious to learn more about that skepticism.

Among the online spaces I sometimes linger, nowhere is these strains of thinking more evident than on Bluesky, the recently-emergent microblogging platform for Twitter/X refugees who (understandably) have no desire to use Meta’s Threads. In my experience, the vibes towards AI on Bluesky have always hovered somewhere between willful ignorance and outright disdain. Mark Cuban, of all people, discovered this in a pretty clear cut way earlier this week when he shared this post:

The responses were what anyone paying attention to the tenor of conversation about AI on Bluesky might’ve predicted. It was basically a lot of this:

To Cuban’s credit, he handled the blowback with a good degree of openness. But the tone of responses to his initial post was so clear that he felt compelled to follow up later, sharing his bemusement:

So what’s going on here? There seem to be a few factors driving AI skepticism, each of them blending into one another in various ways. Posts like the responses that Cuban got on Bluesky tend to display one or more of the following, in my view:

Subscribe now

• Politically-minded distrust: left-leaning people who associate AI with the newly-rightward leanings of Silicon Valley and hate it by association (this could also be called an anti-billionaire/anti-Tech/anti-VC dimension); people in this bucket don’t dismiss AI’s technological potential— in fact they acknowledge it— and some worry about job loss for humans and the rich getting richer and poor getting poorer from it

• An anti- non-deterministic software dimension: folks who cite hallucinations from LLMs and bizarre media outputs (photos, videos, songs, etc.) as evidence that AI is fundamentally flawed as an approach to building useful software; these folks dismiss AI wholesale because they think it’s nothing but smoke and mirrors

• A moral dimension: people who are angry that models have been trained without permission from creators, rights holders, etc. and are opposed to using them at all as a result

• A fool-me-once dimension: people who think AI is simply the latest hype cycle for VCs and rich people to pump before they dump, having made their money and leaving behind a landscape of unfulfilled promises; while overlapping with politically-minded distrust, this dimension seems distinct to me, and is rooted more in a cold skepticism than a political worldview

If you accept that there’s some truth to each of these as motivations for AI skepticism (and maybe you don’t!), there’s a lot to unpack here. Some thoughts on the first two buckets:

Politically-minded distrust

For one, it does seem to me that AI as a field, an idea, a bogeyman, or even just a meme now finds itself firmly engulfed in the culture wars. And while it’s certainly not true that all people on the left are AI skeptics (👋), it does seem to be true that many of the strains of AI skepticism seem to have a leftward ideological underpinning.

Interestingly, though (and to play devil’s advocate against this purely vibes-based observation), I don’t see a lot of evidence that the converse is true: I don’t know that right-leaning people tend to be more enthusiastic about AI than those on the left. In fact, if you look at the political party breakdowns for the YouGov data I mentioned earlier, there doesn’t seem to be an overwhelming political divide at all:

So what to make of what seems to be politically-oriented AI skepticism emanating from the left? I think there are two possibilities:

1. I’m wrong and it’s not very common. Anyone who knows me well knows my penchant for making terrible predictions— it’s why I try not to make many of them. It’s definitely possible that I’m just viewing the world through an extremely potent filter when reading Bluesky. (That much, in fact, is certainly true.)

2. I’m right and it just hasn’t shown up in the national data yet. Saying this dynamic only shows up through a filter bubble doesn’t discount the existence proof that each of Cuban’s negative replies provide. The sentiment is definitely real— it’s just a question of how common it is. My strong suspicion is that there’s growing momentum behind AI skepticism on the left given the cultural and political moment we find ourselves in, and I’d be surprised if the numbers don’t eventually show it.

Anti- non-deterministic software

This is a clumsy name for what I think is the most interesting dimension driving AI skepticism. What’s fascinating to me about is that it seems grounded in a view that the outputs from software and computers are supposed to be objective, logical, predictable and correct in some sense by their very nature.

That, of course, is just not true of generative AI— hallucinations are real, mistakes are real, poor quality outputs of all types are real. So these folks aren’t wrong. But they do seem unwilling to accept a world where we engage with computers more like we engage with other people: knowing that what we hear from people may have inaccuracies, be driven by ulterior motives, etc., but at the same time having a corpus of signals to fall back on that helps us make sense of that information nonetheless: is this person credible overall? Are they credible on this topic? What might they want me to do with this information? How much credence should I give it as a result?

The shift from deterministic interactions with software to probabilistic ones is profound, and for some people, one bad experience with an LLM or other type of model or tool will damage the credibility they ascribe to AI for a long time. Compounding this challenge is the fact that, as with any tool (or as with interacting with people!), it takes trial and error, experience, and developing your own set of heuristics to understand how best to use AI. Without a willingness to engage, experiment, and learn, many people will see bad outputs and discount the whole field forever. Ethan Mollick referred to this gap recently as “the jagged frontier”:

Last week, I wrote about a paper that showed evidence that people who tend to avoid uncertainty are more likely to project feelings of “creepiness” onto technology that seemingly knows more than the person wants it to about them. Those feelings tend to feed into a broader pattern of distrust and skepticism towards those types of technology and, sometimes, lead to conscious decisions to avoid engaging with them altogether. I wonder if a similar pattern could play out with AI skepticism.

It’s good to have healthy skepticism of new technologies, of course, especially during hype cycles like the one we’re in with AI. And any new technological wave creates ripples of people who reject it outright, question its value, or want it to play no role in their lives. That’s all fair and even good. My suspicion, though, is that the value of AI is so profound, and it will eventually reach into so many disparate parts of our lives, that skeptics will face a choice that many other massive technological waves like the internet, smartphones, or social media have forced us all to make: accept its role in your life in some measure, or miss out on a difficult-to-measure-but-obviously-very-real set of benefits.

All of this, of course, would make great fodder for real research instead of me spouting nonsense in a commentary post. Perhaps we’ll do just that in a future research project from The Understanders. In the meantime, I’d love to hear your take: what am I missing about AI skepticism? What research have you seen about it? I’m ready, as always, to be proven wrong on any of this stuff.

Follow The Understanders on LinkedIn

Importantly— and because the research was done before the mass adoption of LLM-powered assistants like ChatGPT, Claude, and Gemini— the term “AI-powered” in this case refers to the machine learning, natural language processing, and other technologies underlying previous-generation assistants like Siri, Alexa, and Google Assistant, and Samsung’s Bixby.

Subscribe now

That cohort is, of course, already in its golden years. Let’s call them the Golden Girls, with no disrespect to queens Blanche, Sophia, Dorothy, and Rose. Google Assistant, bless her heart, still has a hard time remembering how to turn on NPR in my kitchen, and still hasn’t gotten over her love of plumbing the Spotify archives to share a cover, remix, or live version of a song instead of the original studio recorded version. And she gets even less capable as time goes on.

But I digress. What, exactly, did the researchers aim to study and understand about our relationships with the Golden Girls? In a word: creepiness.

Although research exists on users' experiences with DAs, less is known about how user characteristics influence perceptions of a ‘dark side’—perceived creepiness.

(Blanche, maybe. But the others? Creepy? Rude.)

The researchers found evidence that three distinct sources of concern influence the extent to which we get creeped out by digital assistants:

1. Privacy concerns: worries about how personal information was being collected and used

2. Technology anxiety: general unease about interacting with AI systems

3. Uncertainty avoidance: a personal tendency to feel uncomfortable with ambiguous situations

Beyond this, they observed that when we find digital assistant technology creepy, we tend to distrust it, and when we distrust it, we tend to disengage with it.

In the authors’ words:

This study analyses the downsides (‘dark side’) of DAs by examining how user characteristics influence users' experiences. The proposed model demonstrates that privacy concerns, uncertainty avoidance, and technology anxiety have significant positive effects on perceived creepiness and distrust of DAs. This distrust leads to disengagement, where users are less likely to interact with DAs. Furthermore, the study reveals uncertainty avoidance acts as a moderator, strengthening the relationships privacy concerns and technology anxiety have with perceived creepiness. Interestingly, the study shows that these user characteristics (privacy concerns, uncertainty avoidance, and technology anxiety) indirectly influence distrust through perceived creepiness. Moreover, uncertainty avoidance moderates this indirect effect of privacy concerns on distrust. This means that for users with high uncertainty avoidance, privacy concerns have an even stronger influence on their feelings of creepiness, leading to greater distrust.

It’s a veritable domino-into-a-flywheel effect for the technology-skeptical: if you’re a person who tends to avoid uncertainty, you’re more likely to have privacy concerns, and if you have privacy concerns, you’re more likely to perceive technology like this as creepy, and if you consider it creepy, you’re more likely to just… avoid it altogether.

If this is how the more skeptical among us feel about our beloved Golden Girls, one can only imagine how those folks will feel about chatting with an Omniscient ScarJo.

Is this finding a bit tautological? Maybe. But nevertheless, it’s scientific evidence of an important truth as we start to see mass adoption of a new generation of exponentially more powerful digital assistants. As AI gets more powerful in the short term, the returns and benefits of using it will likely increase, yet the gap between adopters and non-adopters may also grow thanks in part to effects such as this one. It doesn’t seem unreasonable to me to imagine that human idiosyncrasies like this may contribute to a new digital divide— one that may make the one we worked hard to address with the internet seem quaint.

Thankfully, American society has collectively agreed that there’s a role for the federal government to play in addressing societal gaps like this, and we have a robust network of agencies staffed by our best and brightest conducting research and running programs to effectively address them!

Sigh.

As always, thanks for reading. I’ll try to dial down the snark next time.

There’s been an interesting lag in the discourse about AI. We’ve become so intrigued by what it’s capable of and how quickly those capabilities are advancing that we haven’t had the time, focus, or inclination to examine the interfaces and products that we’re using to engage with it.

But it feels like something shifted just in the past week on that front. First we had Casey Newton at Platformer talking about DeepSeek’s user experience as a competitive differentiator and the implications for AI lab product teams. Then something of a meme popped up this past weekend about the model selection UI in ChatGPT. Here’s Benedict Evans’ post about it, but I saw others (including Dare Obasanjo and, again, Casey Newton) share similar screenshots and observations, too.

It’s an interesting question: do most users of ChatGPT feel this way? How common is model switching as a behavior, and how well do people understand it? It’s the perfect candidate for our first piece of original research.

In partnership with Outset, we recruited 51 English-speaking paid subscribers to ChatGPT in the US, UK, and Australia to participate in an AI-moderated, semi-structured interview about this exact topic. It was my first experience using Outset, and it was a pretty powerful one: the participants were all sourced and interviewed in under a day on the platform, and it produced a summary report in a matter of minutes. (I used the automated reporting to direct my own synthesis, but it was a huge help nonetheless.) All told, we completed the study in three days. (Note that any video clips were from participants who granted consent for their interviews to be shared without their names attached online.)

Research Questions

We wanted to know:

1. How aware are paid users of the ability to switch between models?

2. For what reasons do they switch models, if they ever do?

3. How confident do they feel in their ability to match their objective to the most appropriate model?

4. What are their overall perceptions of the ability to switch between models?

Here’s what we found, and— in a bit of unforgivable and shameless clickbait— the interesting paradox we uncovered:

Findings

Subscribe now

Read more

YouGov’s own writeup of the findings paints a fairly negative view on how Americans feel about artificial intelligence. And there is definitely evidence in the data to support that framing.

About one-third (36%) of Americans are very or somewhat concerned that artificial intelligence will cause the end of the human race on earth — fewer than in April 2023. Among people who know a great deal about AI, 49% are concerned about this.

Subscribe now

As far as concerns about the implications of a new technology go, that one seems… nontrivial. There are other red flags, too. Among a balanced set of words describing how one could feel about AI, “cautious” and “concerned” topped the list.

And there’s data about how many Americans trust AI to be ethical (only 27%), unbiased (only 34%), and truthful (an interesting 49%).

But looking at the topline data and crosstabs (which I encourage you to do) reveals a more interesting and nuanced reality, like the fact that there’s a nearly perfect split between the percentage of Americans who think AI’s effects on society will be positive (35%) and those who think they’ll be negative (34%).

And zooming into the effects that the technology will have on their own lives, people’s attitudes actually skew more optimistic. A third of Americans believe AI will have a positive effect on their life, compared to 21% who believe it will have a negative effect.

That optimism seems to be at odds with other results from the survey. Attitudes about the perceived risk of job loss, ethical behavior of AI companies, and the government’s ability to meaningfully regulate the space all show much higher measures of skepticism.

You could make an argument that this is a pretty clear-cut case of optimism bias. People tend to exhibit some measure of dissonance between the risks of an event at large and the risks of that event affecting them. Sure, bad things happen, but I don’t think bad things will happen to me.

It’s a revealing truth about where we are in our relationship with AI, and one that resonates with my personal feelings. I can (and do) intellectually recognize the risks of AI’s capabilities at both the micro and macro levels, yet I still get excited when I see a new way to incorporate them into my life or work. Snapshots like this one are a good reminder of that reality, and a good reminder to pump the brakes a bit before we find ourselves getting out over our skis.

Personas are often bullshit. But that’s not what’s interesting right now.

What’s interesting right now is a new paper that explores how using AI personas might affect designers across dimensions like empathy for the persona, understanding the needs of the persona, and their ideation outcomes. The authors turned personas like the one outlined in the image above (minus a photo, to remove potential demographic bias) into interactive characters powered by GPT-3 and had participants conduct design tasks using both the interactive, AI-powered persona and a more traditional, static persona.

The results were mixed. Measures of understanding and empathy weren’t statistically different between the traditional persona use and the AI persona use, but designers did come up with more varied understandings of the persona.

While designers showed comparable diversity in the ideas that emerged from both conditions, we find in the Synthetic User condition a greater variation in how designers perceive the persona’s attributes. We also find that the Synthetic User allows novel interactions such as seeking feedback and testing assumptions.

So, basically, a tossup with maybe a slight advantage given to the AI persona.

Hilariously, the authors point out that part of what held designers back in using the AI-powered personas was treating them as research subjects and employing proper qualitative research techniques to their interrogations of them:

Designers are trained to conduct interviews with users… and to be careful to not influence the interviewee’s mental processes; they are encouraged to employ “neutral prompts” in order to be non-leading and non-directive, such as “Tell me about …” and “How do you feel about …” (Wilson, 2013). This creates a dichotomy in thinking modes– the intuitive, emergent thinking of designers versus the deliberate, structured prompting needed to effectively engage with LLMs (Cross, 2023). Our study results suggest that this clash may limit the effectiveness of Synthetic Users in iterative, constructive dialogues, essential in the design process.

This nugget in itself, while a conjecture of the authors and not a finding per se, is interesting. It suggests that the observed effects may have been greater had the participants been more fluent or advanced in their use of LLM-powered chatbots to begin with. So add that to the standard requirements in your next product designer job description :)

While I don’t plan on delving too deep into meta discussion of user research as a field in this newsletter, I’ll use this paper as an excuse to stray from that rule for a bit, because it explores one expression of a really important and broader idea: the possibility of doing research to understand real people by interviewing artificial ones. This concept has caused a mini-freakout among user researchers in the last couple of years as a startup called, aptly, Synthetic Users popped up on the scene.

Although the idea may seem absurd on the face of it, and while there is certainly no binary yes-or-no answer as to whether it’s a good idea for any and all research questions, there has been some evidence that synthetic users hold promise as a research tool. An amazing paper from last year took a rigorous look at how it might work to a certain degree of effectiveness.

The researchers interviewed 1,000 people about their lives and fed the transcripts of those interviews into an LLM, creating 1,000 synthetic users. Those synthetic users were then given a barrage of standard attitudinal tests like the General Social Survey and the Big Five Personality test. The humans took the tests, too, but twice. This let the authors compare the synthetic users’ ability to predict “accurate” survey responses (which vary even among the same person over time) to the actual humans’ own recall of their own responses. In the General Social Survey, the synthetic users did that with 85% accuracy 🤯

While the study about designers using AI personas is interesting and thought-provoking, the design of this latter study seems to make a more compelling case for how synthetic users may impact social science research of all types in the future. Of course, the questions in the GSS aren’t reflective of every type of research question, and the accuracy of an approach like this will be highly dependent on the context of a particular study and what a researcher hopes to learn. But a wide-ranging, two-hour, in-depth interview like the one powering the synthetic users in this second paper is obviously far more data-rich and therefore more powerful than what’s included in a user persona— a tool that’s reductive by design. These synthetic users almost surely would have fared better in the first study than the ones the researchers used.

The implications here are pretty profound, I think. It could mean that researchers can use one sample of many long interview transcripts to conduct hundreds of studies rather than just one. It could be a massive efficiency, unlocking a new speed of doing social science or user research.

But when the users of AI are researchers or designers, all of the same dimensions of trust, understanding, and influence that affect consumer uses are still at play, of course. Researchers, too, are prone to falling back on wobbly heuristics of trustworthiness, and the ease of doing new research without suffering the pain of recruiting real life participants will be a siren song to many of them. It will be easy for them to jump headfirst into this world and throw caution to the wind. But being vigilant about sample and participant quality will still be critical in a world of doing research on synthetic users. Doing so will just look very, very different.

And with that comes a whole slew of new things to study and learn about how we interact with and feel about them. A new paper takes an interesting look at this by studying the preferences people have for the AI voice assistants they use. Among the findings (note that the sample was comprised exclusively of students at an American university):

• Nearly half of participants (43%) took the time to switch the “embodied voice interface” (EVI) used in voice assistants

• They generally preferred EVIs with American and British accents; British accents, however, were more trusted

• “Participants preferred EVIs that matched their characteristics”

I didn’t pay for the whole paper, so I’m only seeing the abstract, but these seem to be in line with what one might expect. The theory of homophily famously suggests that birds of a feather tend to flock together. It stands to reason that people may exhibit a stated preference for a voice that sounds like someone like them in a voice assistant.

Beyond the observations about user preference with respect to EVIs, the paper also makes some interesting observations about trust. Seemingly independent of the EVI used, it notes:

Most trusted their VAs’ information accuracy because they used the internet to find information, reflecting inadequate mental models… A significant correlation was found between the participants’ perceived intelligence of their VAs and trust in information accuracy.

This called to mind a massive paper released last year from Google DeepMind on “The Ethics of Advanced AI Assistants.” The paper— a whopper at 200+ pages before its footnotes— explores a huge range of dimensions along which AI assistants should be viewed, built, and evaluated, with a whole section focused on human-assistant interaction where they explore topics like influence, trust, and anthropomorphism, where they observe:

When anthropomorphic features are embedded in conversational AI, its users demonstrate a tendency to develop trust in and attachment to AI (Skjuve et al., 2021; Xie and Pentina, 2022)

and:

Users who interact with DVAs with realistic voice production capabilities exhibit a concerning inclination to generalise purely human concepts to digital assistants (Abercrombie et al., 2021). When a DVA’s simulated voice mimics a ‘female’ tone, for example, people ascribe gendered stereotypes to their DVAs (Shiramizu et al., 2022; Tolmeijer et al., 2021)… This evidence suggests that, once initial impressions of human-likeness have been established, the process of anthropomorphism extends beyond context-specific instances and instead permeates broadly to evoke a wide range of human-like attributions.

We’ve already started down this path, where we are ascribing distinctly human traits and characteristics to different models. The added layer of a dynamic, customizable, and human-like voice with which to interact with them makes people even more prone to being influenced by them in ways that they don’t fully understand.

One of the lessons from this new paper and the existing body of research referenced by DeepMind is that interfaces matter. The voice your AI assistant uses will influence how trustworthy you think it is, and will therefore shape when and how you use it and how you feel about it. I suspect that the colors, font, and animation styles (and, as we’ve seen in other research, use of citations) of chatbots have similar effects.

The downstream implications of these hundreds of small design decisions are massive. People are already falling in love with chatbots— a phenomenon that will likely be set aflame as advanced voice models become more and more effective and common across apps and tools.

This is surely one of the most important aspects of AI that doesn’t get sufficient levels of attention in the current discourse. There seems to be a desire to laugh at or dismiss the idea of human-AI relationships, but I think it will not only be a wildly common phenomenon, but also one of the most important areas of study as we try to understand what our evolving relationship with technology looks like.

In the same way that people are infinitely inscrutable— where we can never fully disentangle the full picture of a person’s motivations, desires, and behaviors and how they influence the world around them in big and small ways— we will likely come to some sort of acceptance of AI being similarly inscrutable and mysterious. And that acceptance of a certain level of mystery may in itself may be the foundation for a whole new type of human relationship with technology.

Subscribe now

Riding in an AV for the first time is a hell of a thing. For most people, it’s one of those rare moments with a new technology that truly inspires awe and encourages you to make a mental note of it— you immediately realize that your life now bridges a massive before-and-after moment for what it’s like to be a human navigating the modern world.

Here in San Francisco, where my kids and I take bets on how many Waymos we’ll see on the walk to school each morning, the enthusiasm for AVs is real. People (myself included) gladly pay a premium over the cost of an Uber for the comfort, reliability, and safety they provide over a human driver. Friends love taking them. They’re a tourist attraction. The Hard Fork guys clearly love them. People can’t wait to try the cute new Zoox AVs currently being tested in the SoMa district. It’s an AV lovefest.

But this data from YouGov is a cold bucket of water to dump on that unbounded enthusiasm of the Bay Area. Most Americans (58%) are not enthusiastic about the development of AVs. There are interesting generational, regional, and gender enthusiasm gaps observed in the data, too: older Americans, women, and those who live in rural areas are most skeptical of them.

And it’s not just a lack of enthusiasm! The survey also included measures of “worry” about AV development that show similar patterns.

And there are certainly people who hate these things, even in San Francisco. My wife saw a guy standing in front of a Waymo in a crosswalk just the other day for seemingly no reason other than to make a statement. Public transit boosters question why we would encourage more cars at all instead of more buses or trains. Some are just distrustful of their safety. Some reporters seem hell bent on casting them as unsafe. There will surely always be a segment of the population that hates them for one reason or another.

Subscribe now

But the YouGov data also shows signs that we’re likely in the early days of an emergent, widespread acceptance of a profoundly important new technology. Nearly all measures of acceptance and enthusiasm were up meaningfully in the survey over 2023.

As a shift in technology skepticism, this seems to me to be a pretty steep trajectory over the course of just a year. And the trend tracks with the experience of anyone who has taken a ride in one with a skeptical friend or family member. I’ll never forget the reaction of my mother-in-law when we took her in her first Waymo ride: the journey she went on from mild nervousness to wonder to utter boredom lasted all of 30 seconds from when the car started moving. I imagine that will be a common experience. People will be nervous about these things until they take a ride in one and see how utterly banal the experience is. Only then will the revelations about their safety, comfort, and predictability make themselves clear.

I’m reminded of when Google Street View was first released decades ago. There was backlash to the fact that images had been captured of people’s faces and placed on the internet in identifiable locations without their consent. Google eventually blurred human faces in Street View in response, but before it became clear that doing so was possible, a lot of technology observers and pundits were questioning the ethics of the product and wondering aloud if it would stay alive in the face of public or even governmental pressure. I had these questions myself, but a coworker quickly dismissed that skepticism with a line I’ll always remember: “Street View isn’t going anywhere; it’s just too amazing.”

Sometimes that’s true. And it’s not the capability of the technology on its own that’s too amazing to dismiss (though the technology is amazing). It’s the value that the technology delivers to people that’s too amazing. Society accepts these things over variable time horizons, but only when the value they deliver is more powerful than the downsides— real or perceived— that come along with them. Whether you’re talking predictability, road safety, passenger safety, comfort, pedestrian safety, or something else, the value AVs deliver is so profound that it’s hard to imagine the trends in this YouGov data doing anything other than continuing in the direction towards widespread acceptance.

Put another way: AVs aren’t going anywhere; they’re just too amazing.

Macro stories like this one, about people’s attitudes towards new technology, and micro stories, about the nitty gritty details of how they engage with them and why, are what I hope to track over time at The Understanders. There’s too much talk of new technology as interesting in and of itself— ooo, what can it do now??— and not enough talk about what those advances actually mean to people, how we’re using them, whether or not they are delivering on their promises. In the case of AVs, I can’t wait to see research that dives into many of the micro parts of this story, among them:

• How does the presence of AVs impact the behavior of human drivers around them on the road?

• How do passengers feel riding in AVs without traditional car configurations (like the forthcoming Zoox “toaster”, and presumably more models like it in the future)?

• To what extent to human drivers assign agency for AV behavior to their passengers?

• To what extent do AV passengers feel and assume agency for AV behaviors?

Our results show that users from creative industries (e.g., designers, artists, and film directors) prefer choices-based input (i.e., Choices2I) due to its flexibility and control, regardless of their expertise level. These users often have clear creative goals and find the structured options helpful in achieving specific outcomes. Conversely, non-creative industry users, who typically use T2I models for entertainment or casual purposes, favor text-based input because it is simpler and imposes less cognitive load without the pressure of making choices.

There’s an interesting paradox when it comes to the empty text composer as a design element, particularly in the context of interacting with generative AI. On the one hand, it seems like the most accessible and understandable user interface imaginable. Want to create something? Just type what you want and it’s yours! On the other, what could be more intimidating than that? Where do you even begin? It’s both the lowest and the highest form of cognitive load to introduce to a user.

Consumer-facing AI image and video apps handle this cold start problem with varying degrees of heavy-handedness. Multimodal tools often offer lightweight starter suggestions for engaging with them, where they suggest a (likely new) user try to create an image by way of showcasing the model’s capabilities, as Meta does at meta.ai.

Further along the spectrum of nudging users towards specific types of prompting is something like what Luma does with their Dream Machine video model, where they have an empty composer, but with hovering prompt starters that encourage experimenting with its capabilities in terms of subject matter, lighting, and camera movement.

And at the opposite end of the spectrum from an empty composer, you have apps like Midjourney offering something similar in spirit (but more robust) to what the researchers did in the above paper. There’s an icon in the composer that reveals multiple ways to adjust the dimensions of your prompt via buttons and sliders.

That Midjourney has moved in this direction on its web product is particularly telling, and seems to lend credence to the findings of the study. Midjourney is a tool that’s used heavily by a passionate community that loves experimenting with the model’s capabilities. That many of its users so actively share their prompts on Discord, X, and elsewhere is strong evidence that taming these models via text only is just, well… super hard.

So I imagine more professional creative AI tools will keep evolving in the direction of structured interfaces like this. But I think that raises some interesting questions that I hope future research will dig into and explore:

• Do structured interfaces impede creativity by overly encouraging constraints? Will users of tools like this come to be overly reliant on dimensions like “Weirdness” as opposed to other dimensions they might explore via text-only or other types of interfaces?

• How do people feel about and understand the relationship between structured interfaces that modify prompts and text-only prompts that accompany or are modified by using them?

Stay tuned.

But, of course, doing that is a different transaction altogether, and one that comes with tradeoffs. Poring over several links from search results can be laborious, but it comes with all sorts of attendant signals about how much you can believe what you’re reading (the rank on the results page itself, the publisher, the author, the sources cited on the page, etc.). Just how and to what extent these signals are used and made sense of varies greatly from person to person, of course, but the signals are there nonetheless. When you ask a question of an LLM, you often get a clear answer but no attendant signals about the quality of that answer.

A new paper explores one tactic LLMs use to address that gap: citations. And I love the study design that was used here: using the same prompt, the researchers randomly returned the same answer to participants with no citations, one citation, or five citations. For the people who got citations in their response, they were delivered either valid or random citations. Participants were then asked how much they trusted the answer.

There were three big takeaways in the results:

1. Trustworthiness of responses increased when citations were present

2. When citations are random, however, trustworthiness of responses was lower

3. Trustworthiness of responses was not meaningfully different if there was one or five citations

So, put simply, citations improve trust in responses, but only when they’re valid, and more is not necessarily better.

The implications here for product builders are interesting and largely rest on the idea of “validity.” The sources shown for the random citation in the study are clearly not relevant to the question at hand— it’s easy for someone to say they don’t trust the NBA as a source of information about space. The validity of source data for less general knowledge queries are presumably much harder to parse for an end user and much easier for a model to get wrong.

If you’re building an enterprise AI layer that seeks to deliver accurate information to an employee about the state of a specific project or deal, for example, how do you identify the best sources of truth and ensure that those are what 1) get referenced by the model in its answer and 2) get presented as reliable, if not canonical answers to the query? Making sure both are done exceedingly well is a prerequisite for using citations as a meaningful signal for trust, and citations should probably be reserved for responses where that level of confidence is very high, otherwise they’ll have the opposite of their intended effect.

I’m starting The Understanders because I love technology, but I think there’s too much talk about what new technology is capable of and not enough talk about about how people use it, feel about it, and understand it.

Luckily for me, I was able to turn my love of technology into a career. If you’re not familiar with the field of user research, it’s the work of applying scientific methods to understand exactly these things. But when you do user research in industry, you do it for an employer or a client and the findings rarely see the light of day outside of the company. Academic research is public, but it’s often too arcane to be useful or interesting outside of its immediate field, too difficult to follow or discover, and just not as present in industry discourse as it could or should be.

My hope for The Understanders is that it will be a useful place for you to follow new, publicly-available research on how people are using and feeling about AI and other new technologies. I’ll offer a mix of reporting and commentary that will hopefully be useful and interesting to you whether you’re a user researcher like me, an academic, a journalist, a product person, or an executive.

The Understanders will start out by offering short, easily consumable summaries of emerging research about how people are using AI and the occasional commentary. I’ll aim for at least one post per week. If it seems like something that’s interesting to people and has legs, I may try other things (like crowdfunding my own primary research studies and making the results available to paying subscribers, or interviewing researchers about their work as a podcast), but we’ll cross that bridge if and when we get there.

In the meantime, I hope you’ll subscribe, engage, and maybe even help me out along the way with your thoughts, arguments, and ideas.