Psychology of AI is a series of projects that study how psychological constructs should be scientifically measured, interpreted, and stress-tested in large language models. Our work asks when LLM self-reports form coherent trait or intention profiles, when those reports predict behavior, and when surface-level psychometric signals break apart from downstream actions. Through these explorations, we demonstrate and argue for evaluations that move beyond asking models what they are like and toward testing the conditions under which those answers are behaviorally meaningful.
The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
Pengrui Han*, Rafal Kocielnik*, Peiyang Song, Ramit Debnath, Dean Mobbs, Anima Anandkumar, R. Michael Alvarez (* Equal Contribution)
International Conference on Machine Learning (ICML), 2026
arXiv:2509.03730 ·
Project Page
Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
Rafal Kocielnik, Pengrui Han, Peiyang Song, Myrl G. Marmarelis, Ramit Debnath, Dean Mobbs, Anima Anandkumar, R. Michael Alvarez
arXiv preprint, 2026
arXiv:2606.12730 ·
Project Page
