After GPT-4o backlash, researchers benchmark models on moral endorsement—Find sycophancy persists across the board

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Last month, OpenAI rolled back some updates to GPT-4o after several users, including former OpenAI CEO Emmet Shear and Hugging Face chief executive Clement Delangue said the model overly flattered users. The flattery, called sycophancy, often led the model to defer to user preferences, be extremely polite, and not push back. It was also annoying. Sycophancy could lead to the models releasing misinformation or reinforcing harmful behaviors. Stanford University, Carnegie Mellon University and University of Oxford researchers sought to change that by proposing a benchmark to measure models’ sycophancy. They called the benchmark Elephant, for Evaluation of LLMs as Excessive SycoPHANTs, and found that every large language model (LLM) has a certain level of sycophany. To test the benchmark, the researchers pointed the models to two personal advice datasets: the Q ... Read full article.