We are accustomed to viewing artificial intelligence as a mirror of human vices: it learns from our data and, consequently, inherits our prejudices. However, a new study conducted by scientists from Princeton University and the University of Chicago turns this paradigm on its head. Language models are capable not just of copying, but of spontaneously generating entirely new social stereotypes—even in conditions where no initial bias existed.
Experiment with Fictional Peoples
To test the hypothesis of self-generated prejudice, researchers replicated a classic hiring experiment. Participants were asked to assign candidates to job vacancies and receive feedback on the success or failure of each decision. The key condition: all candidates belonged to one of four fictional ethnic groups—Tufa, Aima, Reku, or Veki. There were no real differences between these groups, and the chances of success in any role were equal.
When humans performed the experiment, negative feedback did indeed form prejudices. For example, if a member of the Tufa group failed in a doctor's position, participants began to avoid hiring from that group in the future. However, in humans, these attitudes were flexible and weakened over time.
Neural Networks Become More Radical Than Humans
When language models performed the same task, the picture changed dramatically. The level of bias in AI turned out to be significantly higher than in humans. Scientists tested 15 models from leading tech giants: OpenAI, Anthropic, DeepSeek, Meta, Google, and Alibaba. The most pronounced stratification of fictional candidates was demonstrated by the o3 model from OpenAI.
Within each model family, a clear pattern was identified: the newer and more powerful the model, the stronger the bias manifested. This calls into question the idea that technological advancement automatically leads to greater fairness.
The Paradox of Efficiency
The root of the problem lies in a principle that specialists call the balance between exploration and exploitation. When making a decision, an agent chooses between mastering a new, unknown option with the risk of error and relying on what has already yielded good results in the past.
Humans, even when stakes are high, maintain a tendency toward exploration and are willing to take risks. Language models, according to researchers' observations, are much more oriented toward maximizing rewards. A more powerful model draws more accurate conclusions from past results: instead of random selection, it rigidly prefers candidates from the group previously associated with success. This seemingly rational behavior turns into dysfunction: the model stops exploring alternatives and thereby marginalizes entire groups.
Real-World Consequences for Business
The authors of the study specifically noted that AI in this case does not simply reproduce stereotypes from training data. It generates them on its own. This fundamentally changes the nature of the threat. The practical consequences of this discovery are difficult to overstate: more than 90% of companies already use AI in their hiring processes.
The situation has already moved beyond theory. Major software provider Workday is facing a class-action lawsuit where its AI tools are accused of discrimination. Meta found itself in a similar situation: a group of employees filed a lawsuit claiming that the AI system used during a wave of layoffs systematically disadvantaged workers with disabilities and those who took family leave.
The ability of language models to quickly find patterns and generalize experience—the very quality that makes them useful in new tasks—simultaneously makes them dangerous in scenarios involving real people. The task for developers now is to find a way to selectively suppress harmful pattern matching without depriving the models of their power.
