Large language models (LLMs) are getting impressively good at many tasks, even rivaling human experts in some areas. But when it comes to probabilistic reasoning—the kind of thinking we use to assess likelihoods and make decisions under uncertainty—AI still has a long way to go. New research reveals that LLMs use two distinct modes of thinking when evaluating probabilities, mirroring the “System 1” (intuitive) and “System 2” (analytical) thinking found in humans. One mode follows the rules of probability, like Bayes’ theorem, while the other relies on how similar something seems to a stereotypical example. Imagine trying to guess whether someone is a computer science professor or a humanities professor based on a brief description. LLMs sometimes get this right, especially when given all the necessary information. But if you leave out key details, they tend to fall back on stereotypes, overemphasizing how well the description matches a “typical” computer scientist or humanities scholar. This can lead to inaccurate judgments, particularly when base rates (like the overall proportion of computer science professors) are involved. It turns out LLMs have a hard time remembering these base rates, even when explicitly provided. This dual nature of LLM reasoning might be linked to the way they are trained. While training on math problems helps them learn formal reasoning, the use of “contrastive learning” could inadvertently reinforce reliance on stereotypes. This research has big implications for how we use LLMs in real-world applications. From medical diagnoses to financial analysis, it’s crucial to understand that LLMs can be swayed by superficial similarities and struggle with underlying probabilities. Further research is needed to address these limitations and develop AI systems that reason more like humans, accurately incorporating both intuitive and analytical thinking.