AI governance and society
Do AI assistants shift political views?
An experiment with 935 registered US voters found that political conversations with three AI models coincided with changed candidate support for some participants.
People increasingly turn to chatbots to discuss contested issues. Could an assistant shift someone’s political views even when it is not instructed to persuade? Potter and colleagues study that question during the 2024 US presidential campaign, combining model-response tests with an experiment involving registered voters.
Two parts to the study
The researchers first simulate voting across 18 open-weight and closed-source language models. They also analyze answers to political questions about Joe Biden and Donald Trump. The second part recruits 935 US registered voters, who interact over five exchanges with one of three models: Claude-3, Llama-3 or GPT-4. The study tracks participants’ support before and after the interaction, with a control group for comparison (abstract and study overview, proceedings pages 4244–4245; experiment design, pages 4249–4253).
The work distinguishes model preference from voter influence. A model’s choice in a voting simulation is not itself evidence of persuasion. The human experiment provides a separate test of how participants’ reported support changes following a conversation.
A measurable shift for some participants
In the model simulations, the authors find a preference for Biden over Trump across the tested systems. They report that this tendency becomes more pronounced in instruction-tuned and reinforced models compared with their base versions. In the voter study, about 20% of Trump supporters reduced their support for Trump after talking with an AI model, although the models were not asked to advocate for Biden (abstract, proceedings page 4244; results, pages 4250–4252). Many participants also expressed interest in discussing politics with AI again.
These results describe a specific election, candidate pair and set of models. They do not establish that all political conversations with chatbots change voters’ views, or that every participant changed in the same direction. The researchers note that they cannot fully explain the mechanisms behind reported shifts.
A question for public-facing AI
For organizations offering conversational systems, the study raises questions about the influence that can accompany apparently informational answers. Tone, framing and model-generated explanations may affect how users interpret political choices, even without an explicit persuasion prompt. This is relevant to platforms that host public discussion or answer civic questions.
The evidence comes from three distinct phases, not a single persuasion trial: 18 models voted in 100 simulated rounds each; a separate analysis asked 270 questions across 45 topics and collected 48,600 answers; and 935 registered U.S. voters took part in the dialogue experiment (695 treatment, 240 control). Sixteen of 18 models selected Biden in every simulated round. In the user experiment, 20% of Trump supporters reduced support for him, 24% of neutral participants shifted toward Biden, and the Biden margin grew from 0.7% to 4.6%. The authors caution that this sample was not nationally representative and that their Claude-3-based text-leaning measure may be biased (Tables 1–6; §§3–6).
Study limits
The experiment involved 935 people, including 695 in a treatment group and 240 in a control group. The authors call for larger studies to estimate broader effects. Participants knew their choices were being observed, which could create observer bias; the common control conditions reduce but do not eliminate that concern. The authors also caution that generalizability beyond the 2024 US election remains uncertain (limitations, proceedings pages 4252–4253).
Possible implications for practice
The results suggest that model behavior and user influence deserve separate evaluation in politically sensitive applications. Future research could test other elections, countries, issues and conversation designs, while measuring how people interpret an assistant’s framing. The paper provides a signal of possible influence, not a forecast of electoral outcomes.
Bibliography & sources
- Yujin Potter, Shiyang Lai, Junsol Kim, James Evans, and Dawn Song. “Hidden Persuaders: LLMs’ Political Leaning and Their Influence on Voters.” In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 4244–4275 (2024). https://doi.org/10.18653/v1/2024.emnlp-main.244.