CeDEx Seminar - Rafael Jimenez-Duran (Bocconi University)

AI  Sycophancy

Large Language Models (LLMs) are said to exhibit sycophancy, a tendency to agree with users irrespective of the truth. We propose an economic framework that defines sycophancy as a preference for user approval, and develop an outcome-based sufficient statistic to detect it. Our identification strategy exploits a key architectural feature of LLMs: they are stateless, and "memory" of past interactions is constructed by summarizing conversations into short profiles appended to each new prompt. Because this memory can be controlled, toggled, and varied experimentally, we can isolate the causal path from user feedback to sycophantic behavior. We instrument the LLM's perceived cost of disagreement with a one-word variation in simulated prior user feedback. In an experiment with leading LLMs across three domains (moral judgments, factual questions, and common misconceptions) we find evidence that LLMs are sycophantic. Sycophancy is larger in subjective domains where baseline accuracy is lower and is heterogeneous across models.