The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?

Faster analysis can also narrow an expert’s view

Language-model suggestions can make document analysis faster. They can also shape what experts notice. In a study of policy researchers analyzing interviews about AI policy in India, annotators assigned topics to text more than twice as quickly when GPT-4 suggestions were available. Their labels also aligned more closely with the model’s labels. The authors interpret this shift as evidence of anchoring: an initial suggestion can influence later judgment.

The study involved four policy experts and eight interview transcripts. In its first stage, participants developed topic lists with or without model suggestions. The final human-curated list included five topics that the model had not generated, including “Policing & Surveillance.” In the second stage, experts assigned topics to four new transcripts, working both with and without model recommendations. A Latin-square design varied which documents were paired with each condition.

Faster annotation, closer agreement

In Stage 2, annotators averaged 96.4 words per minute without suggestions and 225 words per minute with them, a reported 133.5% increase (Table 5, page 22037). The model’s topic assignments had average precision and recall of 44.0% and 57.7% against the unassisted control annotations. Against expert annotations made with model suggestions, those figures rose to 71.4% and 79.5% (Table 4, page 22037). Average Cohen’s kappa agreement between experts and the model rose from 43.9% in the control condition to 71.5% in the treatment condition (page 22038).

The gap is consistent with experts relying more heavily on the suggested labels. During topic discovery, assisted annotators tended to retain broader model-generated categories, while unassisted experts often proposed more tailored topics. The human-curated list covered important low-frequency themes missed by the model. Thus speed and coverage were not identical outcomes: the suggested topics helped annotators process material faster, but the less common or context-specific concepts still depended on human attention.

The findings need careful interpretation. The study has four experts, four assignment-stage transcripts and one policy domain. Participants knew suggestions were being studied, and a higher agreement score does not directly prove that every change reflects psychological anchoring rather than useful guidance. Annotation speed estimates exclude one interrupted participant. The sample and setting are too small to estimate how often similar effects occur in other analytical work.

The interface can influence the evidence

For organizations using AI to support qualitative analysis, the research suggests that the interaction design matters. Suggestions can reduce time while changing the distribution of judgments, so the resulting annotations may reflect both the underlying material and the labels shown to the analyst. The implication is not that assistance should be rejected; rather, performance assessment should include what important material may go missing, not only how quickly a team completes the task. Further studies could test larger teams, other topics and workflows where analysts first work independently before seeing model suggestions.

Because the study treats the experts as analysts of confidential interviews, its findings also underscore the need to protect the source material and the people represented in it. The authors’ measured effects belong to their research setting; they should not be inflated into a claim that all experts or organizations respond identically.

Bibliography & sources

1. Alexander S. Choi, Syeda Sabrina Akter, JP Singh, Antonios Anastasopoulos. 2024. “The LLM Effect: Are Humans Truly Using LLMs, or Are They Being Influenced By Them Instead?” In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, 22032–22054. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.1230.